i^/M 


\?1X  '775  ..✓VS^ 


Office  Of  The  Under  Secretary  Of  Defense 
(Acquisition  and  Technology)  Director, 

Test  Systems,  Engineering  and 
Evaluation  /Test  and  Evaluation 


-islcrsca  I 


mm  Qfummn 


JOINT  TEST  AND  EVALUATION 
HANDBOOK 


1996 


Approved  by 

Office  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology)  Director, 
Test  Systems,  Engineering  and  Evaluation/Test  and  Evaluation 


19960807  024 


JOINT  TEST  AND  EVALUATION 
HANDBOOK 


1996 


Approved  by 

Office  of  the  Under  Secretary  of  Defense  (Acquisition  and  Technology)  Director, 
Test  Systems,  Engineering  and  Evaluation/Test  and  Evaluation 


19960807  024 


FOREWORD 


The  Joint  Test  and  Evaluation  (JT&E)  Program  has  been  established  to  evaluate  concepts  and 
address  needs  and  issues  that  occur  in  joint  military  environments.  The  program  includes  the 
nomination  process;  the  Joint  Feasibility  Study  (JFS),  which  determines  whether  selected 
nominations  are  needed  and  feasible;  and,  when  such  a  determination  is  made,  the  execution  of  a 
JT&E  by  an  Office  of  the  Secretary  of  Defense  (OSD)  chartered  Joint  Test  Force  (JTF). 

DoD  Directive  5000.3-M-4  describes  the  JT&E  program  process,  identifies  the  principle 
participants  and  their  responsibilities,  and  outlines  the  framework  within  which  each  Service 
supports  the  program.  Service  support  is  described  in  detail  in  applicable  Service  directives  and 
the  Memorandum  of  Agreement  on  Multi-Service  Operational  Test  and  Evaluation  and  Joint  Test 
and  Evaluation  which  is  maintained  by  the  Services'  Operational  Test  Agencies  (OTA). 

This  handbook  is  one  of  three  that  discuss  testing  in  the  joint  environment:  the  Joint  Test  and 
Evaluation  Nomination  Handbook,  the  Joint  Feasibility  Study  Handbook  and  the  Joint  Test  and 
Evaluation  Handbook.  These  companion  volumes  are  designed  to  provide  consolidated  OSD 
guidance  and  direction,  information,  references,  and  procedures  to  joint  test  directors  and  their 
staffs.  These  handbooks  provide  "how  to"  lessons  learned  based  primarily  upon  inputs  from 
personnel  who  have  participated  in  JFSs  and  JTFs  in  various  capacities. 

The  proponent  of  this  handbook  is  the  Deputy  Director,  Test  and  Evaluation. 

Recommended  changes  or  suggestions  for  additions  should  be  forwarded  to; 

Deputy  Director,  Test  and  Evaluation 

3010  Pentagon,  Room  3D1080 

Washington,  DC  20301-3010 


Richard  R.  Ledesma 
Deputy  Director 
Test  and  Evaluation 
OUSD  (A&T) 
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CHAPTER  1 

JOINT  TEST  AND  EVALUATION  PROGRAM  BACKGROUND 


A.  INTRODUCTION 

The  Department  of  Defense  (DoD)  Joint 
Test  and  Evaluation  (JT&E)  program  originated 
in  response  to  the  1970  Blue  Ribbon  Defense 
Panel  Report.  The  panel  report  concluded  that 
"There  has  been,  and  is  currently,  no 
effective  means  for  conducting 
productive  joint  operations  tests  and 
evaluations.  The  fact  that  some  such 
efforts  heretofore  have  encountered 
difficulties  and  achieved  few  useful 
results  does  not  obviate  the  requirements 
for  much  needed  Joint  operational  test 
and  evaluation  (OT&E)."  The  report 
recommended  that  continuing  responsibility  for 
joint  testing  be  vested  in  an  Office  of  the 
Secretary  of  Defense  (OSD)  staff  element 
dedicated  to  test  and  evaluation  (T&E).  While 
the  original  emphasis  was  on  joint  operational 
T&E,  evolving  multi-Service  needs  and  issues 
have  led  to  an  expansion  of  the  effort  to  include 
joint  developmental  and  operational  T&Es.  The 
JT&E  program  is  directed  by  the  Director,  Test, 
Systems  Engineering  and  Evaluation 
(D,T,SE&E). 

The  Deputy  Director,  Test  and  Evaluation 
(DDT&E)  manages  and  administers  the  JT&E 
program  and  develops  the  means  to  ensure  that 
productive  testing  is  accomplished.  The  policies 
and  details  in  the  JT&E  program  are  set  forth  in 
DoD  5000.3-M-4,  Joint  Test  and  Evaluation 
Procedures  Manual.  The  JT&E  Procedures 
Manual  provides  a  description  of  the  JT&E 
program,  OSD  and  Service  responsibilities 


relative  to  the  program,  and  defines  the  JT&E 
nomination  and  selection  process.  Key  to  the 
management  structure  is  the  independent  status  of 
the  OSD  JT&E  program.  That  is,  the  program  is 
structured  so  that  it  minimizes  the  influence  of 
Service  biases.  While  JT&E  activities  are 
conducted  in  accordance  with  established  joint 
procedures,  consistency  is  maintained  with  the 
assigned  mission  of  each  participating  Service. 
Responsibility  for  conducting  each  JT&E  is 
assigned  to  a  lead  Service.  The  JT&E  is 
supported  by  personnel  and  resources  from  the 
designated  participating  Services. 

B .  PURPOSE 

The  purpose  of  a  JT&E  is  to  accomplish 
one  or  more  of  the  following: 

•  Assess  the  interoperability  of  Service 
systems  in  joint  operations  and  explore 
potential  solutions  to  identified  problems. 

•  Evaluate  and  provide  recommendations  for 
improvements  in  joint  technical  and 
operational  concepts. 

•  Develop  and  validate  system  development 
and  testing  methodologies  having  multi- 
Service  application. 

•  Evaluate  technical  and  operational 
performance  of  systems  under  realistic  joint 
operational  conditions. 

The  purpose  of  a  Joint  Test  Force  (JTF)  is 
to  plan  and  execute  the  time-sequenced  test 
activities  in  the  approved  Analysis  Plan  for 
Assessment  (APA),  produce  valid  data,  analyze 
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the  data,  and  use  the  analysis  results  to  address 
the  JT&E  concepts  or  issues. 

C .  JT&E  PROGRAM 

The  JT&E  program  is  composed  of  three 
separate  but  closely  related  activities: 

1)  The  nomination,  coordination,  and 
approval  of  a  proposed  JT&E. 

2)  A  joint  feasibility  study  (JFS)  to  determine 
the  need  of  a  proposed  program  test  design 
for  an  approved  nomination. 

3)  The  execution  of  those  JT&Es  that  show  a 
potential  for  significant  improvements  in 
joint  capabilities. 

The  activities  in  the  JT&E  program  are 
sequentially  accomplished  with  reviews 
conducted  at  designated  milestones  to  assure  that 
OSD  and  Service  resources  are  wisely  expended 
and  are  supportable.  These  activities  are 
designed  to  be  expanded  in  both  scope  and  level 
of  detail  as  information  and  expertise  are 
developed,  thus  increasing  confidence  in 
program  decisions  at  the  designated  milestones. 
For  this  reason,  the  Joint  Test  Director  (JTD)  and 
staff  must  understand  the  entire  JT&E  program  to 
be  able  to  execute  the  JT&E  in  accordance  with 
the  APA  and  collect  and  analyze  the  data  required 
to  address  the  JT&E  issues. 

The  JT&E  program  begins  with  a  request 
for  nominations  from  the  Director,  Test,  Systems 
Engineering  and  Evaluation  (D,T,SE&E)  to  the 
Services,  Commanders  in  Chief  (CINC), 
Defense  Agencies,  the  Joint  Staff,  and  the  OSD 
staff.  This  request  is  made  with  the  stipulation 
that  nominations  be  forwarded  to  the  D,T,SE&E 
by  early  March.  The  CINCs,  Services,  and 


Defense  Agencies  develop  and  refine  their 
nominations  and  forward  them  in  the  format 
prescribed  in  DoD  5000.3-M-4.  Nominations 
must  address  concepts  or  issues  that  involve 
more  than  one  Service  and  contain  an  explicit 
statement  of  the  JT&E  purpose,  with  expected 
results,  and  the  issues  to  be  addressed. 

Upon  receipt  of  the  nominations,  a  JT&E 
Planning  Committee  (PC)  will  be  convened  to 
review  and  winnow  the  nominations  and  prepare 
the  highest  priority  candidates  for  SAC  review. 
The  make-up  and  responsibilities  of  the  JT&E 
PC  are  outlined  in  DoD  5000.3-M-4.  The  joint 
staff  member  of  the  JT&E  PC  will  provide  joint 
exercise  information  during  nomination  review 
and  coordination. 

The  JT&E  PC  Service  representatives  will 
exchange  information  on  Service  positions, 
coordinate  on  projected  resource  requirements, 
and  assist  in  the  preparation  of  a  nomination 
report  that  is  presented  to  the  Technical  Advisory 
Board  (TAB)  and  the  Senior  Advisory  Council 
(SAC).  DoD  5000.3-M-4  also  outlines  the 
membership,  duties,  and  responsibilities  of  the 
TAB  and  SAC.  The  SAC  reviews  the  JT&E  PC 
reports  and  develops  reeommendations  as  to 
which  candidates  should  be  approved,  and 
prioritizes  which  candidates  should  proceed  to  a 
JFS  or  directly  to  a  JT&E  for  planning  and 
execution.  Based  on  the  SAC  recommendations, 
the  D,T,SE&E  may  approve  the  candidates  for 
execution,  designate  the  lead  and  participating 
Services,  and  either  direct  a  JFS,  or  charter  a 
JT&E  without  benefit  of  a  JFS. 

Under  the  oversight  of  OSD,  the  lead 
Service  will  conduct  a  JFS  to  expand,  and  refine 
the  candidate,  assess  the  need  and  feasibility  of 
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executing  a  JT&E,  develop  a  APA,  and  prepare 
recommendations  to  the  SAC  and  the  D,T,SE&E 
relative  to  potential  courses  of  action.  The  SAC 
will  consider  the  JFS  report  and  recommend  to 
the  D,T,SE&E  whether  or  not  the  proposed 
JT&E  should  be  chartered  for  execution.  If 
chartered,  a  JTF  will  be  established  and 
organized  to  plan  and  execute  the  activities 
identified  in  the  APA,  analyze  the  results,  and 
evaluate  the  JT&E  concepts  or  issues. 

Although  a  JFS  conducted  by  a  joint 
Service  team  is  an  integral  part  of  the  JT&E 
program,  there  are  special  circumstances  that 
involve  deviations  to  the  normal  sequence  of 
activities.  These  deviations  could  impact  the 
availability  of  personnel  and  resources  to 
transition  from  the  JFS  to  the  JTF  and  the  actions 
and  activities  that  will  be  required  of  the  JTF. 
The  following  are  three  deviations  to  the 
sequential  JT&E  process  that  the  JTD  should  be 
aware  of: 

•  D,T,SE&E  may  elect  to  fund  an 
existing  joint  activity  to  conduct  the 
JFS.  This  would  occur  if  candidate  concepts 
or  issues  were  imbedded  within,  or  closely 
related  to,  concepts  or  issues  being  addressed 
by  an  existing  joint  activity.  This  approach 
may  require  additional  Service  personnel 
augmentation  to  provide  the  required  skills  to 
fully  examine  the  feasibility  of  the  concept  or 
issues  in  a  three-year  effort.  In  the  event  that 
this  type  of  JFS  leads  to  the  chartering  of  a 
JT&E,  the  JTD  can  expect  that  only  the 
additional  military  and  civilian  personnel  who 
supported  the  JFS  will  be  considered  for 
transition  to  the  JTF. 


•  D,T,SE&E  may  elect  to  employ  a 
Federally  Funded  Research  and 
Development  Center  (FFRDC)  or  an 
independent  contractor  to  conduct  the 
JFS.  In  this  case  the  Services  would  provide 
select  personnel  to  work  with  the  FFRDC  or 
contractor  to  ensure  that  Service  concerns  are 
adequately  addressed.  In  the  event  this  type  of 
JFS  leads  to  the  chartering  of  a  JT&E,  the  JTD 
can  expect  that  only  the  Service  personnel  who 
participated  in  the  JFS  will  be  considered  for 
transition  to  the  JTF. 

•  D,T,SE&E  may  elect  to  charter  a 
JT&E  directly  without  first  conducting 
a  JFS.  Although  rare,  this  may  occur  if 
D,T,SE&E  decides,  in  concert  with  a  SAC 
recommendation,  that  the  candidate  involves 
time-sensitive  concepts  or  issues,  is  not  overly 
complex,  can  be  resolved  through  a  JT&E 
effort  of  less  than  three  years,  and  the 
candidate  and  supporting  rationale  are 
developed  to  an  extent  that  satisfies  the  intent 
and  requirements  of  a  JFS.  In  this  case, 
the  JTD's  immediate  task  will  be  to 
acquire  the  resources  to  fully  develop 
the  concepts,  issues,  objectives,  plan 
of  action,  milestones,  test  concept, 
APA,  and  estimates  of  resources  that 
would  normally  be  developed  during 
the  JFS.  In  this  situation,  the  only  personnel 
that  could  be  considered  for  transition  to  the 
JTF  would  be  those  associated  with  the 
preparation  and  development  of  the  candidate. 

The  JT&E  TAB  is  an  organization  of  senior 
civilian  scientists  and  engineers  from  OSD  and 
the  Services  that  provides  advice  to  the  SAC,  the 
OSD  sponsors,  and  the  JTD  on  technical  issues 
related  to  the  JT&E  program.  DoD  5000.3-M-4 
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outlines  the  membership,  duties,  and 
responsibilities  of  the  TAB.  The  TAB  chairman 
also  serves  as  the  technical  advisor  to,  and  is  a 
nonvoting  member  of,  the  SAC.  The  JTD  will 
present  test  plans,  test  results,  and  reports  to  the 
TAB  for  technical  review,  comments,  and 
recommendations.  Read-ahead  copies  of 
presentations  should  be  provided  to  the 
voting  members  of  the  TAB  prior  to 
formal  TAB  presentations. 

If  the  JT&E  involves  technically  complex 
or  difficult  concepts  or  issues,  the  JTD  should 
consider  the  establishment  of  a  technical  advisory 
group  (TAG)  to  provide  additional  technical 
assistance  to  the  JTF.  DoD  5000.3-M-4  outlines 
the  procedures  for  establishing  a  TAG  as  well  as 
their  duties  and  responsibilities. 

If  the  JT&E  involves  concepts  or  issues 
that  pertain  to  policy,  doctrine,  or  tactics,  the  JTD 


should  consider  the  option  of  establishing  a 
General  Officer  Steering  Committee  (GOSC). 
The  GOSC  would  consist  of  flag  officers  (1-2 
star  level)  from  the  Services  who  would  advise 
the  JTD  on  Service  policy,  doctrine,  tactics,  and 
roles/missions  related  to  the  JT&E.  The 
organization  and  roles  of  the  GOSC  members 
would  be  dictated  by  the  particular  objectives  of 
the  JT&E.  If  established,  the  JTD  should  include 
the  GOSC  in  the  review  cycle  of  JT&E  planning, 
analysis,  and  preparation  of  reports. 

The  JTD  should  remember  that  the  purpose 
of  the  JT&E  is  to  address  and  evaluate  concepts 
or  issues.  The  JT&E  is,  therefore,  an 
objective  evaluation  of  the  concepts  or 
issues  and  the  JTF  must  not  become  an 
advocate  of  any  particular  concept  or 
solution  to  an  issue.  JTF  ANALYSIS 
MUST  BE  BUILT  ON  THE  PREMISE 
OF  OBJECTIVITY. 


1-4 


JT&E  Handbook 
February  1,  1996 


CHAPTER  2 

ESTABLISH  AND  ORGANIZE  THE  JTF 


A.  INTRODUCTION 

This  chapter  explains  the  relationship  of  the  JTF 
to  the  JT&E  program  and  describes  those  actions 
that  must  be  accomplished  upon  charter  to  obtain 
facilities,  personnel,  and  administrative  support. 
The  chapter  also  addresses  the  functional  and 
organizational  requirements  that  should  be 
considered  in  establishing  and  organizing  the 
JTF.  The  JFS  final  report  and  the  APA  provide 
the  foundation  for  the  JT&E.  The  Feasibility 
Study  Director  (FSD)  will  have  established  and 
coordinated  initial  JTF  requirements  for  facilities 
and  personnel  with  the  host  installation  and  the 
Service  personnel  centers.  These  actions  should 
provide  the  JTD  with  a  facility  to  work  from 
initially  and  a  core  staff  of  personnel  who  make  a 
transition  from  the  JFS  to  initiate  actions  for  the 
conduct  of  the  JT&E. 

B.  INITIAL  ASSISTANCE 

One  of  the  first  tasks  of  a  newly  appointed 
JTD  will  be  to  contact  the  lead  Service, 
participating  Service,  and  OSD  JT&E  points  of 
contact  (POC)  to  discuss  in  detail  requirements, 
available  support,  reporting,  and  expectations. 
Annex  C  is  a  list  of  the  JT&E  POCs. 

When  the  JTF  is  chartered,  the  lead  and 
participating  Service  headquarters  will  issue  an 
order,  letter,  or  directive  that  designates  the 
subordinate  command  or  agency  that  will 
function  as  their  support  agent. 


The  JT&E  nomination  process  involves  a 
number  of  planning  and  coordinating  activities 
that  relate  directly  to  JTF  requirements.  If  the 
JT&E  is  chartered  without  the  benefit  of  a  JFS, 
the  JTF  will  have  to  accomplish  those  program- 
level  planning  functions  associated  with 
development  of  the  APA.  In  this  case,  the  JFS 
Handbook  is  required  reading  for  the  JTD  and 
staff  and  the  JTF  must  obtain  and  become 
knowledgeable  of  all  documents  developed 
during  the  JT&E  nomination  and  approval 
process.  As  a  minimum,  the  following 
documents  should  be  obtained  from  the  lead 
Service  headquarters: 

JT&E  nomination  documents 
contain  the  description  of  the  JT&E  in  terms  of 
the  problem,  concepts,  issues,  objectives,  and 
their  utility  and  impact  on  joint  effectiveness. 
These  documents  identify  the  anticipated  users  of 
the  JT&E  results  and  provide:  a  proposed 
program  schedule/  milestones;  the  identification 
of  unique  hardware,  software,  environment, 
personnel,  and  instrumentation  requirements;  an 
outline  of  a  test  concept;  an  estimate  of  required 
funds  and  resources;  known  or  projected 
shortfalls;  and  agencies,  organizations,  and 
CINCs  concurring  in  the  nomination.  Some 
nomination  documentation  have  been  more 
complete  than  others.  In  any  case,  these 
documents  are  the  starting  point  for  the 
development  of  a  JT&E  concept  and  the  APA. 
These  documents  will  provide  the  JTD  and  staff 
with  an  understanding  of  the  concepts  or  issues 
as  seen  through  the  eyes  of  the  nominating 
command  or  agency. 


JTdcE  Handbook 
February  1, 1996 


2-1 


The  JT&E  planning  committee  (PC) 
identifies  opportunities  for  incorporation  of 
JT&E  activities  into  already  scheduled  joint  or 
Service  exercises,  documents  the  ability  of  the 
Services  to  provide  the  necessary  personnel  and 
facilities,  establishes  the  utility  of  JT&E  results, 
and  estimates  the  impact  of  these  results  on  joint 
capabilities.  The  JT&E  PC  meeting  minutes  will 
also  identify  the  major  participants  in  the  JT&E 
and  the  users  of  joint  test  products  who  must  be 
included  in  the  coordination  process. 

The  SAC  reviews  those  candidates  that 
are  recommended  for  approval  and  lists  those  that 
should  proceed  directly  to  a  JT&E.  The  SAC 
recommendation  for  approval  of  a  JT&E  is 
Service  and  Joint  Staff  concurrence  of  the 
candidates  issues  and  objectives  and  is  an  initial 
commitment  that  Service  support  will  be 
available.  This  concurrence  will  become 
important  during  the  coordination  of 
documents  and  recommendations  as  each 
new  coordinator  has  a  tendency  to  want 
to  restructure  the  program  and  revise  the 
issues  and  objectives.  It  is  important 
that  program  concerns,  issues, 
objectives,  personnel,  equipment,  and 
costs  be  solidified  early  in  the  JT&E 
effort.  The  SAC  minutes  may  be  key  to 
achieving  this  goal.  The  SAC  priority  is 
indicative  of  the  importance  of  the  JT&E  to  DoD 
in  the  event  that  budget  reductions  occur  and 
alternative  considerations  are  required. 

Service  directives  and  instructions 
relating  to  JTF  support  are  written  by  the 
headquarters  staffs  of  the  lead  and  participating 
Services  to  subordinate  commands  or  agencies 
outlining  responsibilities  and  priorities  for 
providing  support  to  the  JT&E.  These  directives 


and  instructions  provide  the  JTD  and  the  Service 
deputies  with  the  authority  to  coordinate  directly 
with  supporting  organizations  to  obtain  facilities, 
administrative  support,  and  staff  personnel. 
These  directives  also  provide  the  designated 
Service  support  agents  with  the  authority  to 
provide  the  required  support. 

Archive  documents  produced  by 
completed  JFSs  and  JTFs  are  maintained  in  the 
Air  Force  Operational  Test  and  Evaluation  Center 
(AFOTEC)  library  at  Kirtland  AFB,  NM.  These 
include  numerous  charters,  directives,  program 
test  designs,  program  test  plans  (PTPs),  test 
reports,  final  reports,  and  management  reports. 
While  some  of  these  documents  may  not  directly 
relate  to  the  chartered  JT&E,  many  problems  the 
JTF  will  encounter  are  systemic  and  have  been 
addressed  and  reported  on  by  previous  JTDs.  Of 
specific  interest  will  be  the  management  reports 
that  contain  the  lessons  learned  and  the 
recommended  improvement/corrective  actions. 
The  library  material  is  available  to  all  JTDs  and 
their  staffs.  It  may  be  beneficial  for  the  JTD  to 
visit  the  AFOTEC  library  at  Kirtland  AFB  to 
review  and  determine  which  of  the  available 
documents  may  be  applicable.  Request  for 
documents  should  be  addressed  to: 

Directorate  of  Research  Service 
8500  Gibson  Blvd.,  SE 
.  Kirtland:  AFB,  NM  87117^5558  :  - 
;ATeiep}iOri&^05|84645g:  'DSN,24^;^i574 


If  the  JTF  is  chartered  as  the  result  of  a 
JFS,  the  JTD  should  transition  all  JFS 
documents  to  the  JTF.  Most  of  the  planning  and 
coordinating  activities  that  occurred  during  the 
nomination  and  approval  process  will  have  been 
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included  in  the  JFS  report  and  the  APA.  In 
addition  to  obtaining  all  JFS  documents,  the  JTF 
should  obtain  the  following  from  the  lead 
Service  OTA  POC: 

JFS  final  report  documents  the  necessity 
and  feasibility  of  the  JT&E,  its  scope,  concept, 
methodologies,  limitations,  risks,  and  resource 
requirements.  The  JFS  final  report  will  also 
contain  the  recommendations  of  the  FSD  relative 
to  the  location,  facility  requirements,  and  staffing 
of  the  JTF.  The  JTD  should  also  obtain  a  list  of 
the  coordination  accomplished  by  the  JFS  relative 
to  JTF  facility  and  resource  requirements. 

APA  defines  and  formalizes  what  actions 
are  required  to  accomplish  the  JT&E,  who  is 
responsible  for  the  required  tasks  and  activities, 
when  and  where  the  test  activities  or  events  will 
be  conducted,  and  what  resources  will  be 
required.  The  specifics  of  how  each  of  the 
JT&E  activities  will  be  accomplished  will  be 
developed  in  a  test  plan  for  each  activity  or  event. 

JTF  charter  is  the  authority  for  the  JTD  to 
proceed  with  staffing,  planning,  execution, 
analysis,  and  reporting  of  the  JT&E.  The  charter 
also  provides  authority  for  the  JTF  to  establish 
direct  contact  with  participating  agencies  to  obtain 
the  required  support. 

C.  INITIAL  REQUIREMENTS 

The  framework  for  Service  support  for 
JTFs  is  DoD  Instruction  5000.2,  Defense 
Acquisition  Program  Procedures,  and  the 
Memorandum  of  Agreement  (MO A)  between  the 
Service  OTAs,  MOA  on  MOT&E  and  JT&E. 
One  of  the  initial  tasks  of  the  newly  appointed 
JTD  should  be  to  contact  the  lead  and 


participating  Service  headquarters,  OTA,  and 
OSD  JT&E  POCs  to  discuss  the  transition  of  JFS 
personnel  and  assets  to  the  JTF,  the  availability 
of  Service  and  OSD  support,  reporting 
requirements,  and  expectations.  Annex  C  is  a  list 
of  JT&E  POCs.  In  the  case  of  the  Navy  and 
Marine  Corps,  the  Service  headquarters  POC  and 
the  OTA  POC  have  been  designated  as  the  same 
person.  Liaison  should  be  established  with  the 
POCs  at  the  Service  headquarters  and  OTA 
levels,  as  both  will  be  involved  in  providing 
personnel  and  resources.  When  the  JTF  is 
chartered,  the  headquarters  of  the  lead  Service 
will  issue  an  order,  letter,  or  directive  that 
designates  the  subordinate  commands  or  agencies 
that  will  function  as  their  support  agents  for 
conduct  of  the  JT&E.  The  headquarters  of  the 
participating  Services  will  issue  an  order,  letter, 
or  directive  designating  subordinate  commands 
or  agencies  to  function  as  their  support  agents 
that  the  JTD  can  call  on  for  assistance. 

When  a  JTF  is  chartered,  the  JTD  will  be 
faced  with  a  myriad  of  requirements  that  include 
establishing  and  organizing  the  JTF,  obtaining 
documents  and  directives,  and  establishing 
liaison  with  POCs.  Initial  emphasis  will  center 
on: 

•  Obtaining  an  adequate  facility. 

•  Obtaining  qualified  staff  and  support 
personnel  in  a  timely  manner. 

•  Obtaining  support  resources 
(communications  and  automated  data 
processing  (ADP)  equipment). 

•  Refining  the  JTF  budget  and  securing  the 
required  funds. 

•  Organizing  the  JTF  and  developing  a  work 
breakdown  structure  (WBS)  and  schedule. 
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•  Establishing  support  and  coordination 
POCs  with  the  Services,  Joint  Staff  and 
OSD. 

•  Providing  for  JTF  security. 

•  Submitting  request  for  resources  to  the 
Services  (See  Chapter  4  for  Details). 

The  JTD  is  directly  responsible  to  the 
Deputy  Director,  Test  and  Evaluation  (DDT«&E) 
for  the  organization,  management  and  execution 
of  the  JT&E.  The  JTD  is  authorized  by  DDT&E 
to  establish  direct  communications  with  Service 
or  DoD  agencies  to  obtain  assistance  or  support 
required  for  the  JTF.  As  a  minimum,  initiating 
the  JT&E  effort  will  involve  establishing  an 
organizational  structure,  and  obtaining  adequate 
facilities,  qualified  staffing,  and  support 
resources.  These  activities  are  discussed  in  the 
following  sections. 

Obtain  Adequate  Facilities 

The  JTF  charter  will  stipulate  where  the 
JTF  will  be  located.  The  lead  Service  is 
responsible  for  providing  the  JTF  with  facilities, 
including  building(s),  furniture,  and 
administrative  equipment.  Facilities  include  such 
things  as  heat,  light,  safes,  computers,  and 
access  to  network  bulletin  boards  and  on-line 
services  (e.g.  MILNET/TECNET).  Facilities  to 
support  deployed  operations  or  detachments  will 
be  arranged  through  negotiations  with  the  host 
installation  at  the  deployed  or  detachment 
location.  It  is  unlikely  that  the  JFS  facilities  will 
be  adequate  to  house  the  fully  staffed  JTF.  The 
JFS  facility,  however,  should  be  adequate  to 
house  the  initial  JTF  cadre,  thus  providing  the 
JTD  time  to  negotiate  with  the  host  installation 
for  the  required  space.  The  FSD  should  have 
developed  an  estimate  of  JTF  facility 


requirements  and  coordinated  these  requirements 
with  the  host  installation.  The  JTD  and  the  lead 
Service  OTA  POC  should  negotiate  with  the  host 
installation  to  obtain  the  required  facilities.  These 
negotiations  will  normally  be  formalized  in  an 
Inter-Service  Support  Agreement  (ISA)  between 
the  JTD  and  the  host  installation. 

Assuming  the  JTF  will  consist  of  40-60 
personnel,  the  JTD  should  acquire  the  use  of 
6000-9500  sq  ft  of  usable  office  floor  space 
(approximately  150  sq  ft  per  person),  preferably 
in  a  single  building.  While  most  of  the  space  will 
be  used  for  offices  and  individual  work  areas, 
one  conference  area  of  at  least  500  sq  ft  should 
be  available  that  could  also  be  used  as  a 
collective  work  area.  The  JTD  should  also 
consider  the  requirement  for  another  1000-1500 
sq  ft  of  separate  space  to  house  secure  automated 
data  processing  (ADP)  equipment  (if  such  is 
required).  Most  JTFs  will  involve  some 
sensitive  or  classified  materials.  If  so,  controlled 
access  to  the  JTF  facility  and  storage  of  the 
classified  materials  must  be  considered.  If  the 
JTF  involves  information  and  discussions  that 
require  access  to  a  Special  Compartmented 
Information  Facility  (SCIF),  this  requirement 
should  be  arranged  through  the  host  installation 
or  OTA  POC. 

Obtain  Qualified  Stafflng 

One  of  the  first  and  most  challenging  tasks 
for  the  JTD  will  be  to  obtain  full  staffing  with 
qualified  and  experienced  personnel  in  a  timely 
manner.  Retention  of  these  personnel  for  the 
duration  of  the  JTF  may  also  be  a  problem  that 
must  be  considered.  Estimated  manning 
requirements  for  the  JTF  were  established  by  the 
JFS  and  were  a  major  factor  in  the  JTF  chartering 
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decision.  When  the  SAC  recommended  approval 
of  the  JT&E,  each  member  on  the  SAC 
committed  his  Service  to  provide  the  required 
resources  to  complete  the  JT&E.  The  Service 
personnel  systems,  however,  can  take  up  to  10- 
14  months  to  provide  permanent  change  of 
station  (PCS)  personnel.  The  JTF  may  have  to 
operate  for  roughly  half  of  its  life  cycle  before 
full  manning  is  provided.  The  JTD,  therefore, 
must  quickly  review  the  manning  requirements 
established  by  the  JFS,  revise  them  as  required, 
and  submit  them  to  the  Service  POCs  for  entry 
into  the  Service  personnel  systems.  The  JTD 
should  recognize  that  even  with  early  JFS 
coordination  of  JTF  manning  requirements  with 
the  Service  personnel  centers,  several  months 
may  be  required  before  PCS  personnel  actually 
arrive  on  station.  In  such  cases,  and  particularly 
if  the  required  skills  are  critical  during  the  initial 
phases  of  the  JT&E,  the  JTD  should  attempt  to 
acquire  personnel  with  requisite  skills  on  a 
temporary  basis  until  the  permanent  personnel 
arrive. 

Designation  as  the  lead  Service  carries  the 
responsibility  of  supporting  the  JTF  with  the 
director,  one  of  the  deputies,  the  majority  of 
Service  personnel.  Civil  Service  personnel, 
administrative  personnel,  and  assistance  in  the 
areas  of  secretarial  support,  travel,  contracting, 
personnel,  administration,  comptroller,  supply, 
and  logistics  (see  DoD  5000.3-M-4).  The 
participating  Services  will  provide  Service 
deputies  and  a  balanced  percentage  of  personnel, 
both  PCS  and  temporary  duty  (TDY),  for  the 
conduct  of  the  JT&E.  Staffing  these 
requirements  depends  on  the  ability  of  the 
Service  personnel  systems  to  respond  to  the  JTF 
requirements.  While  the  procedures  for  staffing 
JTFs  are  being  established,  it  is  sometimes 


difficult  for  the  Services  to  provide  qualified 
personnel  in  a  timely  manner.  Available,  rather 
than  fully-qualified  personnel,  are  sometimes 
assigned  to  these  positions.  The  JTD  should 
insist  on  the  assignment  of  fully-qualified 
personnel  and  all  should  be  on  a  full-time  basis  at 
the  JTF  location. 

The  JTF  will  transition  through  three 
distinct  phases.  Each  will  involve  unique  tasks 
with  unique  personnel  requirements.  The  three 
phases  are  planning,  executing  the  planned 
activities,  and  selling  the  JTF  developed 
products.  The  planning  phase  will  require 
personnel  with  expertise  and  experience  in  the 
mission  field,  in  test  planning,  and  the 
identification,  coordination,  and  acquisition  of 
test  resources.  The  test  execution  phase  will 
require  personnel  with  operational  and  technical 
expertise  and  experience  in  test  execution,  data 
collection,  and  data  analysis.  The  selling  phase 
will  require  personnel  with  expertise  in  report 
writing  and  the  preparation  and  presentation  of 
briefings  to  senior  management  personnel.  The 
JTD  should  be  prepared  to  refine  JTF  manning 
requirements  as  the  JTF  matures  through  these 
three  phases. 

The  time  allowed  for  completion  of  a  JT&E 
precludes  delays  in  obtaining  personnel,  the 
training  of  unqualified  personnel,  or  the 
inefficiencies  inherent  in  the  use  of  part-time  or 
additional-duty  personnel.  If  the  Service 
personnel  systems  cannot  accommodate  JTF 
personnel  requirements  in  a  timely  manner,  the 
JTD  should  consider  the  use  of  qualified 
contractor  personnel  to  support  the  JTF. 
Contractor  support  can  be  obtained  by  either 
modifying  existing  contracts  or  by  using  the 
DESA  OMNIBUS  support  contract.  Either  will 
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involve  the  development,  coordination,  and 
approval  of  a  task  statement  specifically 
addressing  the  JTF  requirements. 

Since  each  JTPs  mission,  scope,  and 
program  is  different,  associated  personnel 
requirements  will  vary.  While  requirements  may 
differ,  past  JTFs  have  revealed  a  series  of 
common  functions  for  which  trained  and 
experienced  personnel  with  the  following  skills 
will  be  required. 

Joint  Test  Director  (JTD)  should  be  an 
0-6  (possibly  a  GS-15)  with  test  and  evaluation 
experience  and  a  background  in  the  technical 
aspects  of  the  JT&E.  Joint  experience  is  desired. 
Command  experience  is  necessary  owing  to  the 
problems  of  establishing  a  new  command.  The 
requirements  to  brief  at  the  flag  level  also  dictate 
that  the  JTD  be  an  0-6.  Senior  0-5s  are 
acceptable  if  they  have  promotion  potential  and 
the  required  T&E  and  technical  experience.  It  is 
expected  that  the  FSD  transition  to  and  become 
the  JTD. 

Program  management  personnel  would 
ideally  be  the  JTD  and  deputy  test  directors 
(DTD)  who  transitioned  from  the  JFS  to  the  JTF. 
If  not,  OSD  and  the  Services  will  nominate, 
select  and  fill  the  JTD  and  DTD  positions.  The 
JTF  management  team  (JTD,  DTDs,  and  Chief  of 
Staff)  should  consist  of  experienced  personnel 
(preferably  0-5/0-6  level)  and  reflect  a  balanced 
mix  of  management  and  leadership  experience, 
joint  experience,  operational  experience  within 
the  JT&E  mission  area,  test  experience,  and 
expertise  in  the  functions  of  the  JTF 
organization.  All  personnel  should  be  respected 
within  the  Services  and  their  field  of  expertise. 
The  Chief  of  Staff  should  be  experienced  in 


program  administration  and  the  JT&E  processes 
and  requirements.  The  Chief  of  Staff  should  also 
serve  as  the  focal  point  of  functional  and  support 
activities  to  ensure  continuity  during  all  daily  and 
test  activities. 

Technical  personnel  with  test  experience 
in  all  areas  covered  by  the  testing  issues  are 
critical.  The  technical  advisor  should  be  a  subject 
matter  expert  in  the  JT&E  mission  area  and 
should  have  experience  in  system  testing, 
analysis  and  statistics.  The  technical  advisor 
should  also  have  experience  in  the  management 
of  technical  teams  as  his  primary  function  will  be 
to  review  and  resolve  technical  differences  of 
opinion  in  order  to  maintain  the  focus  of  the  JTF 
test  and  analysis  activities  on  the  program  issues. 

Test  management  personnel  will  be 
required  with  experience  in  planning  and 
directing  operational  tests,  particularly  field  tests. 
They  should  be  knowledgeable  of  operational 
and  logistical  factors  associated  with  the  conduct 
of  such  testing.  Personnel  best  qualified  for 
these  positions  are  those  who  have  performed 
test  management  functions  with  one  of  the 
Service  OTAs,  Service  tests,  multi-Service 
operational  tests  and  evaluations  (MOT&Es),  or 
JT&Es. 

Military  operations  personnel  will  be 
required  who  possess  knowledge  and  experience 
in  the  operational  aspects  of  the  JT&E  subject 
matter.  These  individuals  should  understand  the 
JT&E  concepts  and  issues  from  a  joint 
perspective  or  at  least  from  the  perspective  of 
their  respective  Services.  They  should  also  be 
versed  in  Service  missions,  doctrine,  tactics,  and 
procedures  that  are  applicable  to  the  JT&E 
operational  issues. 
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Test  and  evaluation  personnel  will  be 
required  who  are  experienced  in  concept 
development  and  test  planning,  execution,  and 
reporting.  This  experience  should  include  the 
development  and  management  of  test  schedules 
and  budgets;  the  identification,  acquisition,  and 
management  of  test  resources;  and  data  collection 
and  management.  These  personnel  should  also 
be  able  to  plan  for  test  analysis,  develop  field  test 
plans  and  procedures,  plan  for  test  execution, 
and  write  test  reports. 

Analysis  personnel  will  be  required  with 
experience  in  operations  research,  systems 
analysis  or  related  scientific  disciplines  and  in  the 
application  of  these  skills  to  the  JT&E.  This 
experience  should  include  developing  test 
designs,  defining  evaluation  criteria,  determining 
statistical  requirements,  identifying  data  analysis 
requirements  and  procedures,  using  automated 
and  manual  analysis  tools,  and  applying 
modeling  and  simulation  techniques  to  JT&E 
analysis  requirements. 

Engineering/Instrumentation 

personnel  may  be  required  with  an  engineering 
background  and  operational  experience  in  the 
identification  and  application  of  various 
instrumentations  employed  in  operational  testing. 
The  experience  required  includes  the  employment 
of  test  systems  and  the  integration  of  test 
instrumentation  system  and  associated  interface 
equipment.  Engineering  experience  may  be 
required  to  identify  the  technical  parameters 
associated  with  obtaining  data  to  satisfy  the  test 
objectives  and  the  types  of  instrumentation  that 
will  be  required  to  collect  these  data.  If  the  JT&E 
involves  field  operations,  persons  with  skills  in 
communications  systems  and  data  transfer  will  be 


required,  including  micro-wave  relays,  cellular 
phones,  radios,  and  radio  relays. 

Intelligence  personnel  will  be  required 
with  training  and  experience  in  threat  analysis 
and  both  tactical  and  strategic  intelligence 
systems  and  employment  procedures.  These 
personnel  will  be  required  for  those  JT&E 
activities  that  involve  test  scenarios  and 
particularly  those  involving  the  employment  of 
threat  simulators  and  systems. 

Database  development  personnel  will 
be  required  with  training  and  experience  in  the 
development  of  databases  and  the  documentation, 
maintenance,  and  configuration  control  of 
databases  and  related  software  products.  These 
personnel  should  also  have  experience  in  test  data 
management,  test  data  reduction,  and  test  data 
analysis  support. 

Resource  management  personnel  will 
be  required  who  are  trained  and  experienced  in 
supply,  logistics,  personnel  administration  and 
budget/fiscal  matters.  While  the  JTF  may  receive 
a  portion  of  this  type  of  support  from  outside 
agencies  (from  the  host  installation),  the  JTF  will 
need  core  resource  management  personnel  for 
day-to-day  operations  and  for  the  coordination  of 
support  from  outside  agencies.  These  personnel 
should  also  have  experience  in  property 
administration,  tracking  procurement  activities, 
transferring  property,  providing  audit  trails  for 
transactions,  and  the  handling  of  shipping/ 
receiving  documents.  The  size  of  this  personnel 
requirement  will  depend  on  the  scope  of  the 
JT&E,  the  size  of  the  JTF,  the  quantity  of 
equipment  to  be  acquired  and  maintained,  and  the 
number  of  JTF  facilities  and  detachments  to  be 
maintained  and  supported. 
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The  tracking,  management,  and  reporting 
of  JT«feE  fiscal  matters  is  extremely  important. 
Therefore,  it  is  important  that  each  JTF  have  a 
financial  manager,  preferably  experienced  in 
OSD  and  Service  budgeting,  funding,  and 
accounting.  The  duties  of  the  financial  manager 
will  include  the  transfer  of  fiscal  authorizations 
and  the  tracking  of  funding  commitments, 
obligations,  and  disbursements. 

In  addition  to  financial  management  and 
pay,  all  JTFs  will  have  purchasing  requirements 
that  range  from  paper  clips  to  computers,  as  well 
as  vehicles  and  other  major  assets  purchased 
through  several  means.  Although  JTF 
military/civilian  pay  will  be  supported  through 
the  lead  and  participating  Service  finance  centers, 
the  JTF  should  have  someone  on  the  staff  with 
expertise  regarding  the  Service  pay  systems. 

Security  personnel  may  be  required  who 
are  trained  and  have  experience  in  physical  and 
operational  security  (OPSEC).  OPSEC 
considerations  can  be  a  major  factor  in  the 
conduct  of  JT&E  activities.  Technical  assistance 
in  the  security  area  may  be  available  through 
DESA  to  include  developing  security 
classification  guides,  establishing  required 
security  levels  for  personnel,  and  obtaining 
personnel  security  billets. 

Administration  personnel  will  be 
required  with  experience  ranging  from  the 
establishment  and  maintenance  of  files  to  the  use 
of  computers  and  word  processing,  graphics  and 
spreadsheet  software  programs.  The  JTF  should 
not  underestimate  this  requirement.  A 
considerable  amount  of  the  JTF  work  will 


involve  the  development  and  maintenance  of  test 
related  documentation  and  the  preparation  of 
reports.  The  JTF  may  also  have  the  requirement 
for  an  editor  and  someone  experienced  in 
computer-based  desktop  illustration  and 
publishing  systems. 

Obtain  Support  Resources 

The  lead  Service  is  responsible  for 
providing  most  support  resources  for  the  JTF  to 
include  furniture  and  administrative  equipment. 
Figure  2-1  is  an  overview  of  Service  and  OSD 
responsibilities  for  providing  JTF  resources. 

Existing  and  suitable  resources  that  can  be  made 
available  from  the  lead  Service  will  be  provided. 
Resource  requirements  that  cannot  be  provided 
from  Service  assets  will  be  brought  to  the 
attention  of  DDT&E.  DESA  may  be  able  to 
provide  some  of  these  resources  from  equipment 
that  was  purchased  and  utilized  by  prior  JT&Es. 
OSD  funds  may  be  provided  to  acquire  unique  or 
special  equipment  required  for  the  JT&E  and 
support  resources  not  otherwise  available. 

The  provision  of  adequate  ADP  and 
communications  equipment  is  the  responsibility 
of  the  lead  Service.  Identifying  equipment 
requirements  is  the  responsibility  of  the  JTD.  The 
JTD  should  establish  and  coordinate  these 
requirements  with  the  lead  Service  OTA  and  the 
OSD  JT&E  Coordinator  early  in  the  program  to 
determine  what  equipment  can  be  provided. 
Considerations  in  developing  these  requirements 
include  when  the  equipment  will  be  needed  and 
for  how  long.  Most  installations  or  agencies 
designated  to  support  the  JTF  will  have  some 
ADP  and  communications  equipment  on  hand 
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4.  Facilities 


5.  Test  Support 
And  Resources 


OSD  SERVICES 


CATEGORY  EXPENSES 


1.  JTD  and  Staff  A.  Travel 

B.  Per  Diem 

C.  Civilian/Military  Pay 

D.  Personal  Computers 

E.  Office  Copier 

F.  Office  Equipment 

G.  Supplies 

H.  Office  Facilities 

I.  Technical  Advisors  (TAB/TAG)(TDY' 

J.  Other  Training  -  Tech,  Staff  (TDY) 

K.  Reports,  Printing  and  Production 


2,  Contractor  A.  Test  Planning,  Tech  Support 

B.  Simulation/Model 


3.  Training  A.  Training  For  Data  Collectors  And  X  (1) 

White  Controllers 

B.  Test  Unique  Doctrine  And  Tactics  X  (1) 

Training  For  Red  Players 

C.  Tactics  Training  For  Blue  Players  X 

4.  Facilities  A.  Buildings  -  Construction,  Lease,  X(l) 

Ateration 

B.  Utilities  X 

5.  Test  Support  A.  Hardware/ADP  X 

And  Resources  B.  Range  Costs  XiT) 

C.  Instrumentation  (Unique)  X(2) 

D.  Surrogates  X 

E.  Site  Preparation  X 

F.  Base  Operating  Support  X 

G.  Player  Communications  Support  X 

H.  Player  Vehicle  Leasing  _ _ X 

I.  PhotoA^isual  Support  [NEGOTIABLE  | 

J.  OPSEC  Survey/Risk  Management  X 

K.  Environmental  Studies  X 

L.  Player  Supplies  X 

M.  Red/Blue  Force  Travel/Per  Diem  X 

N.  White  Force  Travel/Per  Diem  X 

O.  Test  Vehicle  Modification  And  X 

Transportation 

P.  Surrogate  Modification  And  X 

Transportation 

Q.  Flying  Hours  X 

R.  Fuel  X 

S.  Parts  And  Transportation  X 

T.  Munitions  X 

U.  Transport  Of  Munitions  X 

V.  Data  Collection,  Analysis,  Store  X 

W.  Industrial  Fund  Flight  Hours  X 

X.  Simulation  Costs,  Modifications, 

_ _ Transportation  _ _ _ X(3) 

1.  Existing  and  suitable  Service  resources,  facilities,  etc.,  which  can  be  made  available  will 
be  provided;  however,  additional  costs  for  JT&E  program  requirements  and  resources 
should  be  negotiated  with  OSD. 

2.  Existing  and  suitable  Service  range/test  facilities  which  can  be  made  available  will  be 
provided;  however,  any  reimbursement  costs  will  be  funded  by  OSD. 

3.  Existing  and  suitable  Service  simulation  capabilities  which  can  be  made  available  will 
be  provided;  however,  other  additional  costs  including  modification,  transportation  and 
maintenance  will  be  funded  by  OSD. 

4.  DESA  may  be  able  to  provide,  otherwise  the  lead  Service  provides. 


NEGOTIABLE 


Figure  2-1,  Responsibilities  for  Providing  JT&E  Resources 
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that  can  be  made  available.  DESA  is  also  a 
potential  source  for  such  equipment.  Equipment 
that  cannot  be  provided  by  the  Services  or  DESA 
must  be  acquired  through  lease  or  purchase  and 
this  can  take  some  time.  In  order  to  preclude 
impacts  on  the  program,  the  JTD  should  establish 
and  coordinate  JTF  ADP  and  communications 
requirements  with  the  Service  OTA  and  OSD 
JT&E  POCs  as  soon  as  possible.  This  will  allow 
the  JTD  to  identify  funds  and  initiate  actions  to 
obtain  equipment  that  the  Services  cannot 
provide. 

ADP  requirements  will  be  based  on  the 
complexity  and  functional  requirements  of  the 
JT&E,  the  size  and  structure  of  the  JTF  and  the 
number  of  operating  locations.  Database 
hardware  and  software  will  be  a  major 
consideration  in  developing  the  ADP 
requirements,  including  storage,  speed  of 
processing,  number  of  users,  and  geographic 
separation.  In  establishing  the  ADP 
requirements,  the  JTD  should  also  consider  such 
things  as  interface  capabilities  with  OSD  and  the 
Services.  The  capability  to  exchange  and 
manipulate  data  between  organizations  or 
agencies  involved  in  the  JT&E  may  be  very 
helpful. 

The  JTD  should  recognize  that  acquiring 
some  of  this  equipment  may  take  considerable 
time.  ADP  equipment  requirements  must  be 
considered  and  acted  on  early  in  the  program. 

Communications  requirements  should 
include  sufficient  telephone  instruments  and  lines 
to  handle  the  expected  number  of  incoming  and 
outgoing  calls.  The  telephone  system  should 
have  a  call  forwarding  capability  and 


a  message  recording  capability  that  can  be 
accessed  from  an  external  telephone  to  listen  to 
the  recorded  messages.  The  telephone  system 
should  also  have  the  capability  for  both  inbound 
and  outbound  commercial  and  DSN  calls.  Both 
secure  and  non-secure  FAX/modem  and  internet 
(MILNET/TELNET)  capabilities  should  be 
considered.  The  JTD  should  recognize  that 
acquiring  telephone  lines  on  a  military  installation 
can  sometimes  take  four  to  six  weeks. 

Funds 

Funding  for  the  JTF  will  be  in  accordance 
with  chapter  251  of  DoD  71 10.1 -M,  Budget 
Guidance  Manual.  More  specifically  DDT&E 
will  disperse  PE  0605804D  funds  to  pay  for 
costs  that  are  unique  to  the  JTF  and  the  Services 
will  fund  for  the  daily  operation  of  activities.  If 
discussions  arise  relative  to  the  funding  of  any 
JTF  activity  or  support,  the  JTD  should 
immediately  contact  the  OSD  JT&E  Coordinator 
for  clarification  and  resolution. 

OSD  funds  will  be  used  to  pay  for 
travel/per  diem  (JTF  staff),  contractor  support, 
and  the  cost  of  specialized  equipment, 
instrumentation  and  facilities  required  to  support 
JT&E  activities.  OSD  will  initially  provide  these 
funds  based  on  the  resource  estimates  that  were 
established  in  the  JFS.  These  funds  may  be 
committed  during  both  the  fiscal  year  of 
appropriation  and  the  next  fiscal  year,  as  they  are 
two-year  funds.  Although  there  are  many 
exceptions  to  the  use  of  all  types  of  funds,  OSD 
funds  are  perhaps  the  most  varied  in  terms  of 
their  administrative  requirements.  The  JTD 
should,  therefore,  work  closely  with  the 
servicing  financial  activities. 
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Service  funds  are  appropriated  for 
expenses  necessary  for  the  daily  Operations  and 
Maintenance  (O&M)  of  JTF  activities.  The  lead 
Service  will  fund  those  costs  associated  with 
facilities,  personnel  and  administrative  support. 
Among  the  items  covered  by  O&M  funds  are 
salaries  for  civilian  personnel,  travel  (other  than 
the  JTF  staff),  transportation,  utilities  and  rents, 
Service-owned  equipment  maintenance,  supplies, 
printing,  reproduction  and  communications 
equipment.  O&M  funds  must  generally  be 
obligated  during  the  fiscal  year  for  which  they  are 
appropriated.  They  do  not  have  to  be  dispersed 
in  that  same  year. 

Although  a  budget  estimate  for  the  JT&E 
was  developed  by  the  JFS,  it  should  be  revised 
by  the  JTD  and  forwarded  to  DDT&E  within  60 
days  following  JTF  charter.  Significant  funding 
beyond  that  which  is  required  for  initial  JTF 
startup  will  be  dependent  on  the  revised  JTF 
budget  estimate.  The  revised  budget  will  include 
cost  estimates  separated  into  the  major  categories 
of  travel,  per  diem,  and  contractor  costs.  The 
budget  revision  will  be  reviewed  by  DDT&E 
and,  after  discussion  with  the  JTD,  will  be 
approved  and  the  funding  provided.  DDT&E 
may  provide  funds,  either  in  full  or  in 
increments,  for  the  conduct  of  the  JT&E.  The 
JT&E,  like  all  DoD  programs,  is  subject  to 
annual  congressional  budget  cuts  and  reduced 
funding.  Variations  in  JT&E  program  funding 
will  naturally  be  felt  by  the  JTF. 

Once  the  JTF  budget  is  approved  and 
funding  is  provided,  the  JTD  must  closely 
manage  JTF  expenditures.  The  JTF's 
commitments  and  expenditures  are  of  particular 
interest  to  DDT&E.  The  JTD  will  provide 
monthly  financial  status  reports  to  DDT&E  to 


include  commitments  and  expenditures  by 
category  and  funding  balance.  The  JTD  is 
responsible  for  operating  within  the  bounds  of 
the  revised  budget.  If  there  are  expenses  that  are 
not  in  the  revised  budget,  the  JTD  must  obtain 
the  concurrence  of  DDT&E  prior  to  committing 
funds  to  them.  Service  approval  will  be  obtained 
for  any  expenses  that  will  be  funded  by  the 
Services. 

The  JTD  should  establish  a  relationship 
with  the  host  installation  comptroller  early  in  the 
program  to  identify  support  needs  for 
reimbursable  programs  and  establish  applicable 
fiscal  accounts.  The  JTD  should  advise  DDT&E 
and  the  appropriate  Service  agencies  of  these 
accounts  so  that  OSD  and  Service  funds  can  be 
directed  accordingly. 

Managers  at  all  levels  in  DoD  track  funds 
by  the  three  phases  in  the  accounting  cycle: 
commitment,  obligation,  and  expenditure.  The 
JTD  and  JTF  financial  manager  must,  therefore, 
understand  the  phases,  the  impact  of  financial 
related  actions  taken  upon  them,  the 
documentation  required  to  execute  the  financial 
programs  and  be  able  to  report  on  the  status  of 
individual  JTF  actions  in  terms  of  the  accounting 
cycle.  The  following  is  a  brief  overview  of  the 
three  phases  of  fiscal  accountability. 

Commitment  is  an  administrative 
reservation  of  funds  showing  an  intent  to  incur 
an  obligation.  Commitments  are  based  on  firm 
requisitions,  purchase  requests,  or  allied 
documents  which  will  require  actual  contracting 
actions,  or  other  authorized  written  evidence  that 
indicates  intention  to  incur  obligations. 
Commitments  involve  a  certification  of  funds 
availability  that  attests  that  funds  are  currently 
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available  to  cover  potential  obligations  and  that 
these  obligations  are  proper  and  valid  charges  to 
the  funds  cited.  Commitments  are  made  against 
funds  available  for  the  period  in  which  the 
obligation  is  to  be  incurred.  Decommitment  of 
funds  should  be  accomplished  to  increase 
available  funds  anytime  a  requirement  no  longer 
exists,  funds  cannot  be  obligated  within  the 
prescribed  time  frame  (fiscal  year  for  Service 
funds  and  two  years  for  OSD  PE  658D4D 
funds),  or  receipt  of  a  credit  commitment 
document. 

Obligation  is  a  documented  transaction 
that  constitutes  a  legal  requirement  for  furnishing 
goods  or  services  and  a  corresponding 
requirement  to  pay  for  those  items  upon 
completion  or  delivery.  An  obligation  is  the 
amount  of  an  order  placed,  a  contract  awarded, 
services  received,  or  similar  transaction  during  a 
given  period  that  will  require  payments  during 
either  the  same  or  future  periods.  Obligations 
should  be  recorded  only  when  documentary 
evidence  supports  the  amount  of  obligation.  All 
obligation  documents  should  be  sent  promptly  to 
the  servicing  accounting  and  finance  officer  for 
recording  once  an  obligation  has  been  incurred. 
An  annual  appropriation  is  available  for 
obligation  only  during  the  fiscal  year  specified  in 
the  appropriations  act.  In  order  to  obligate  an 
annual  appropriation,  the  contract  must  be  made 
within  the  fiscal  year  to  be  charged  against,  the 
supplies  or  services  must  serve  a  bona  fide  need 
of  that  fiscal  year,  and  the  obligation  cannot 
exceed  the  available  allotment.  Obligations  may 
be  incurred  at  any  time  during  the  period  of  a 
multiple  year  appropriation  if  the  related  action 
concerns  an  item  authorized  in  the  appropriation 
act  and  the  obligation  does  not  exceed  the 
approved  allotment. 


Expenditure  occurs  when  an  invoice  or 
similar  document  has  been  received  and  payment 
has  been  made  against  it  to  satisfy  an  obligation. 
Accrued  expenditures  are  charges  to  an 
appropriation  during  a  period  that  requires  funds 
either  be  dispersed  or  set  aside  for  payment. 
Expenditures  accrue  regardless  of  when  cash 
payments  are  made  and  are  recorded  based  on 
documentary  evidence  including  bills  of  lading, 
contractor  invoices,  and  obligating  documents.  A 
recordable  accrued  expenditure  includes  the 
amount  of  work  accomplished  in  spite  of  the 
stage  of  completion,  shipping  status,  or  lack  of 
acceptance  by  the  Government. 

The  difference  between  the  funding 
received  by  fiscal  year  and  the  portion  that  has 
been  set  aside  as  committed,  obligated,  or 
expended  is  the  available  balance  left  to  conduct 
operations  during  the  lifetime  of  the  funds.  It  is 
important  to  remember  that  the  combined  total  of 
the  available  balance,  prior  expenditures,  current 
obligations,  and  commitments  can  not  exceed  the 
authorized  funding  by  source  and  appropriation. 
More  importantly,  it  is  a  violation  of  public  law 
(Anti-Deficiency  Act)  if  expenditures  and  legal 
obligations  exceed  the  available  funding  of  each 
separate  funding  document.  Additionally,  funds 
cannot  be  commingled,  i.e.,  OSD  and  Service 
funds  accounted  for  as  one. 

Given  the  fluctuations  that  will  most  likely 
occur  in  the  JT&E  program  budget  and,  in  turn, 
the  JTF's  budget,  the  JTD  must  be  prepared  for 
less  funding  than  requested  each  year.  A  major 
consideration  in  budget  cuts  that  a  JTF  may 
incur,  is  the  JTF's  progress  and  success  to  date 
in  the  conduct  of  the  JT&E  and  the  Services’ 
perception  of  the  JT&E's  ability  to  satisfy 
Service  needs.  If  there  is  an  appearance  that  the 
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JT&E  will  contribute  little  to  the  needs  of  the 
Services,  Joint  Staff,  or  OSD,  it  is  likely  that  the 
JT&E  will  be  subjected  to  more  strenuous  budget 
cuts.  For  this  reason,  it  is  important  for  the  JTF 
to  gain  and  maintain  Service,  Joint  Staff,  and 
OSD  interest  in  the  JT&E  and  the  product(s)  that 
will  be  produced. 

Establish  an  Organizational  Structure 

While  most  organizational  structures  can  be 
tailored  to  accommodate  a  JTF,  a  simple  stmcture 
based  on  JTF  functions  will  probably  work  best. 
The  selected  organizational  structure  should 
provide  a  clear  chain  of  responsibility  and 
accountability  for  each  function,  activity,  and 
task.  Figure  2-2  is  an  example  of  such  an 
organizational  structure.  The  organizational 
structure  should  be  tailored  to  take  into  account 
the  unique  nature  and  technical  complexity  of  the 
JTF.  The  JTD  has  the  ultimate  responsibility  for 
the  successful  completion  of  the  JT&E  on  time 
and  within  budget.  The  JTD  is  a  purple-suited 
position  and  he  or  she  must  maintain  both  an 
OSD  and  joint  perspective  of  the  JT&E.  The 
Service  deputies  are  dual-hatted  positions  and 
should  serve  as  both  the  senior  representatives 
from  their  Services  and  as  directors  of  JTF 
functional  divisions. 

Someone  within  the  JTF  should  function  as 
the  focal  point  for  resolving  differences  in 
technical  ideas  and  opinions  in  order  to  maintain 
the  focus  of  JTF  efforts  on  the  chartered  mission. 
This  can  be  accomplished  by  the  designation  of  a 
technical  advisor.  In  some  cases,  the  JTD  may 
function  as  the  technical  advisor.  The  JTD  must 
recognize,  however,  that  immersion  in  the 


technical  aspects  of  the  JTF  could  be  at  the 
expense  of  management  functions. 

A  Chief  of  Staff  position  should  also  be 
considered.  The  Chief  of  Staff  could  function  as 
the  deputy  test  director  during  JTD  absences  and 
should  be  responsible  for  JTF  administrative 
functions  to  include  the  preparation  and  oversight 
of  JTF  reports  and  briefings.  The  JTD  will 
report  to  DDT&E  and  will  be  assisted  by  the 
Service  deputies. 

The  specific  responsibilities  of  the  JTD, 
Service  Deputies,  Chief  of  Staff,  and  technical 
advisor  are: 

JTD 

•  Establish,  organize,  and  supervise  the  JTF 
team. 

•  Lead,  manage,  and  supervise  all  facets  of 
the  JTF. 

•  Ensure  the  JTF  is  conducted  to  accomplish 
its  mission  on  schedule  and  within  budget. 

•  Develop  a  JTF  work  breakdown  structure 
(WBS)  and  schedule. 

•  Develop,  maintain,  and  update 
requirements  for  JTF  funding  and  Service 
support. 

•  Submit  these  requirements  to  DDT&E  and 
the  Services  as  appropriate. 

•  Control  funds  designated  for  JTF  activities 
and  account  to  DDT&E  for  their  use. 

•  Monitor  Service  expenditures  related  to  the 
JTF. 

•  Prepare  and  submit  reports  as  appropriate, 
including  in  process  reviews,  a  JTF  final 
report,  and  a  JTF  management  report. 

•  Market  JTF  products  if  they  have  merit. 
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Figure  2-2.  Example  of  an  Organizational  Structure 


Service  deputies 

Support  the  JTD  in  the  conduct  of  the 
JT&E 

Ensure  that  own  Service  concerns  are 
adequately  addressed  by  the  JTF. 

Identify  Service  resource  and  personnel 
expertise  required  for  execution  of  the  JTF 
and  ensure  their  availability. 

Coordinate  within  own  Service  to  obtain 
required  information  and  resources  and 
obtain  Service  coordination  on  JTF 
documents  and  products. 

Provide  advice  and  expertise  to  the  JTD  on 
Service  issues  and  areas  of  interest. 

Obtain  own  Service  coordination  relative  to 
JTF  planning,  conclusions, 
recommendations,  and  products. 

Provide  progress  and  significant  action 
reports  to  own  Service. 


Chief  of  staff 

Maintain  coordination  chain  and  POC  list 
for  JTF  produced  documents. 

Oversee  JTF  administrative,  fiscal, 
logistics,  and  personnel  functions. 

Prepare  financial  and  progress  reports  to 
DDT&E. 

Prepare  and  coordinate  JTF  documents  and 
reports. 

Prepare  and  coordinate  JTF  briefings. 
Distribute  read-ahead  copies  of  briefings  to 
responsible  agencies. 

Technical  advisor 

Provide  technical  advice  to  the  JTD,  DTDs, 
and  directors  of  functional  divisions. 

Effect  technical  interface  with  TAB 
members. 
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•  Resolve  technical  differences  of  opinion 
relative  to  JT&E  activities. 

•  Conduct  technical  reviews  of  test  plans  to 
assure  they  focus  on  the  JT&E  concepts 
and  issues. 

•  Conduct  technical  reviews  of  all  JTF 
analysis,  findings,  conclusions,  and 
reports. 

Program  Security 

Most  JTFs  will  involve  access  to  or  the 
production  of  sensitive  or  classified  materials.  If 
so,  the  JTD  must  establish  procedures  for  the 
complete  security  of  JTF  functions.  These 
procedures  should  include  consideration  for  the 
establishment  of  controlled  access  areas;  the 
positive  identification  and  control  of  personnel  in 
the  controlled  access  areas;  the  receipt,  storage, 
transmittal,  reproduction,  and  destruction  of 
classified  documents;  communications  and  ADP 
security;  the  control  of  classified  containers  and 
the  container  combinations;  personnel  security 
clearances;  and  facility  security  checks.  Service 
security  manuals,  DoD  5220.22-M  and  DoD 
R5200.1-R  provide  detailed  information  relative 
to  the  Government  Information  Security  Program 
and  the  minimum  requirements  to  prevent  the 
unauthorized  disclosure  of  sensitive  or  classified 
information.  The  JTD  should  consider  the 
requirement  for  the  development  of  a  JTF 
security  plan  that  addresses  all  aspects  of  security 
for  JTF  activities. 

The  JTF  may  require  access  to  Sensitive 
Compartmented  Information  (SCI).  If  so,  the 
JTD  must  ensure  that  such  billets  are  available 
when  needed  either  through  an  existing  Special 
Security  Office  (SSO)  or  DESA  to  support  the 
requirement. 


Similarly,  if  access  to  substantial  amounts 
of  intelligence  information  is  required,  the  JTF 
should  establish  an  account  with  the  Defense 
Intelligence  Agency  (DIA)  for  that  purpose.  For 
access  to  crypto  information,  a  communications 
security  (COMSEC)  account  with  the  National 
Security  Agency  (NSA)  should  be  established. 
Finally,  if  JTF  generated  material  could  be 
provided  to  foreign  governments,  the  JTD  should 
appoint  a  foreign  release  officer  to  assure 
compliance  with  existing  exchange  agreements. 
Guidance  and  assistance  in  these  areas  can  be 
obtained  by  contacting  the  OSD  JT&E 
Coordinator. 

Another  consideration  associated  with 
classified  or  sensitive  projects  is  that  of  marking 
newly  created  documents.  The  classification  of 
these  documents  or  portions  of  the  documents 
must  be  based  on  the  information  they  contain  or 
reveal.  Classification  authority  will  normally  be 
derived  from  original  documents.  If  the  JTF 
involves  the  development  of  classified  material, 
and  it  is  deemed  appropriate,  OSD  will  appoint 
the  JTD  as  an  Original  Classification  Authority 
(OCA).  Upon  JTF  close  down,  the  OCA  should 
assure  that  this  authority  is  passed  to  OSD  for  all 
documents  where  OCA  was  exercised  so  that 
future  classification  issues  can  be  addressed. 

Each  Service  has  its  own  unique 
requirements  for  security  classification  guides 
(SCGs).  The  JTF  should  use  Service  unique  or 
establish  joint  security  classification  guides  when 
possible.  An  outline  for  a  SCG,  which  can  be 
adapted  to  meet  the  needs  of  a  JTF,  is  in 
Annex  J. 
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CHAPTER  3 

JOINT  TEST  PLANNING  AND  EXECUTION 


A.  INTRODUCTION 

Once  an  initial  JTF  cadre  has  been 
established  and  the  required  actions  initiated  to 
acquire  permanent  facilities  and  support 
resources,  the  JTD  must  direct  JTF  efforts 
towards  the  planning  and  execution  of  the  JT&E. 
If  the  JTF  was  chartered  without  a  JFS, 
the  JTD  should  develop  an  APA  (see  Chapter  2 
in  the  JFS  Handbook  relative  to  those  actions 
required  to  develop  an  APA)  in  addition  to  those 
JT&E  planning  and  execution  actions  outlined  in 
this  chapter.  If  the  JTF  was  charted  as  the 
result  of  a  JFS,  JTF  planning  should  focus  on 
those  actions  required  to  plan,  coordinate  and 
execute  those  test  activities  outlined  in  the  APA, 
analyze  the  data  collected,  resolve  the  JT&E 
issues  and  develop  the  JT&E  products. 

This  chapter  provides  an  overview  of  those 
activities  and  actions  required  to  plan  and  execute 
a  JT&E  that  was  chartered  as  a  result  of  a  JFS. 
Part  One  addresses  program  level  planning.  Part 
Two  addresses  specific  test  activity  planning  and 
execution,  and  Part  Three  addresses  program 
reporting  requirements. 

PART  ONE 

JOINT  TEST  &  EVALUATION 
PLANNING 

Joint  testing  is  a  program  of  interrelated  and 
interdependent  test  activities  and  events  that  are 
planned  and  conducted  to  address,  resolve,  or 
satisfy  specific  program  issues  and  objectives. 
The  JFS  developed  an  APA  that  outlines  what 


actions  and  resources  are  required  to  accomplish 
the  JT&E.  JTF  planning  requirements  will  center 
on  defining  the  test  activities  and  events  in  the 
APA  in  terms  of  how  they  will  be 
accomplished,  refining  the  specific 
resource  requirements  for  each  and 
specifying  how  the  data  will  be  collected, 
aggregated,  and  analyzed  at  the  program 
level  to  address  or  resolve  the  issues  or  achieve 
the  objectives.  JTF  planning  to  satisfy  these 
requirements  will  be  incorporated  into  a  program 
level  document.  The  primary  difference  between 
the  APA  and  the  program  level  document  will  be 
in  the  level  of  detail  resulting  from  more  detailed 
planning  for  the  conduct  and  execution  of  the 
JT&E  activities. 

Documentation  of  the  JTF  planning  can  be 
accomplished  in  a  number  of  ways.  The  most 
common  is  to  expand  the  level  of  detail  in  the 
APA  into  a  Program  Test  Plan  (PTP).  A 
suggested  outline  for  a  PTP  is  contained  in 
Annex  L.  The  PTP  should  be  an  extension  of  the 
APA  and  address  those  planning  items  or  actions 
that  have  been  significantly  changed  or  revised. 
It  is  not  the  intent  of  the  suggested  PTP  outline 
that  the  PTP  be  duplicative  of  items  sufficiently 
covered  in  the  APA.  The  program  level  Data 
Management  and  Analysis  Plan  (DMAP)  can  be 
developed  either  as  an  annex  to  the  PTP  or  as  a 
stand-alone  document.  If  the  JTD  considers  the 
APA  to  be  of  sufficient  detail  to  satisfy  JTF 
program  level  planning  requirements,  he  or  she 
may  elect  to  document  JTF  planning  in  a  program 
level  DMAP.  A  suggested  outline  for  a  program 
level  DMAP  is  contained  in  Annex  L.  In  rare 
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cases,  the  JTD  may  elect  to  document  JTF 
planning  in  the  individual  site  or  event  test 
plans/DMAPs.  This  option  would  only  be  used 
when  the  APA  (and  data  management  and 
analysis  methodology)  is  in  sufficient  detail  to 
satisfy  program  level  requirements  relative  to  the 
aggregation  and  analysis  of  all  test  activity  data. 
The  JTD  will  obtain  DDT&E  approval  if  JTF 
planning  is  documented  in  any  form  other  than  a 
PTP. 

The  factors  to  be  considered  in  JTF  program 
level  planning  are  discussed  in  the  remaining 
parts  of  this  chapter. 

B .  STUDY  AND  FULLY  UNDER¬ 
STAND  THE  JFS  FINAL  REPORT, 

THE  APA,  AND  APPLICABLE  JFS 

DOCUMENTS 

An  initial  step  in  JT&E  planning  is  to  review 
and  understand  the  documents  that  led  to  the 
chartering  of  the  JTF.  Section  B  in  Chapter  1 
describes  the  documents  that  should  have  been 
transitioned  to  the  JTF  as  soon  as  the  JTF  was 
chartered.  If  these  documents  are  not  readily 
available,  the  JTD  should  request  them  from  the 
OTA  JT&E  POCs.  The  JFS  final  report  and 
APA  were  the  primary  documents  that  the  SAC 
and  D,T,SE&E  relied  on  for  information  in 
making  their  decision  to  charter  the  JTF.  A 
major  factor  in  the  chartering  decision  was  the 
JFS  conclusion  that  the  test  activities  and  events 
in  the  APA  were  technically  sound  and  that  the 
resources  to  accomplish  them  would  be  available. 
Thus,  the  APA  is  the  foundation  for  JTF 
planning.  Personnel  involved  in  JTF  planning 
must  fully  understand  the  JFS  documents, 
specifically  the  APA  and  the  coordination 
involved  in  its  preparation. 


If  the  JTF  core  staff  is  made  up  of  personnel 
that  transitioned  from  the  JFS,  they  will  be 
familiar  with  the  APA  and  related  documents.  If 
not,  the  JTD  and  core  staff  must  become 
completely  knowledgeable  of  the  concepts, 
issues  and  objectives  in  the  JFS  final  report  as 
they  provide  the  JTF  planners  with  a  real  world 
view  of  the  situation  or  problem.  The  JTF 
planners  must  retain  an  unbiased  and 
joint  focus  on  the  situation  or  problem 
and  the  doctrine  and  policies  associated 
with  the  JT&E.  JTF  planners  must  not 
become  advocates  of  the  proposed 
solution  in  the  APA. 


The  WBS  should  be  coupled  to  the  JT&E 
schedule  to  identify  milestone  reviews  at  key 
points  in  the  program.  The  JTD  will  coordinate 
major  changes  in  the  JT&E  that  occur  between 
the  scheduled  milestone  reviews  with  DDT&E. 
Figure  3-2  is  an  example  of  a  JTF  WBS  with 
suggested  milestones  scheduled  for: 

•  A  DDT&E  review  of  the  JTF  facilities, 
organization,  and  WBS. 

•  An  annual  TAB/DDT&E  review  of  JT&E 
progress. 


C .  DEVELOP  A  JT&E  MANAGEMENT 

PLAN  AND  SCHEDULE 

The  JTD  should  develop  a  Work  Breakdown 
Structure  (WBS)  and  schedule  for  the  JT&E  that 
is  based  on  the  required  activities  and  the 
availability  of  personnel  and  resources.  The 
format  and  content  of  the  WBS  will  vary 
depending  on  the  JTF  missions  and  its  technical 
complexity.  Figure  3-1  is  an  example  of  a 
WBS  for  a  JTF. 
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3  Years 


JTF  ACTIVITY 
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Tailor  Activities/Allocate  times 

•  Technical  Complexity 

•  Available  Staff 


Figure  3-1.  Example  of  a  JTF  WBS/Schedule 


•  A  TAB/DDT&E/SAC  review  of  the  JT&E 
final  report. 

The  JTD  should  forward  the  WBS  and 
JT&E  schedule  to  DDT&E  within  60  days 
after  charter  for  approval.  Reviews  and  progress 
of  the  JT&E  will  be  based  on  this  schedule.  The 
JT&E  schedule  developed  by  the  JFS  was  based 
on  best  available  information  at  the  time  and 
could  change  considerably  as  the  JTF  develops 
and  coordinates  the  plans  for  specific  test 
activities.  Changes  to  the  JT&E  schedule  should 
be  anticipated  and  coordinated  with  DDT&E  as 
they  occur.  The  combination  of  the  WBS  and 
schedule  of  JT&E  activities  and  milestones  is  the 
JTF  management  plan. 


D .  VALIDATE  OR  REFINE  THE 

JFS  COORDINATION  CHAIN 

The  JFS  should  have  determined  and 
established  a  coordination  chain,  identified  the 
POCs,  and  determined  who  has  signature 
authority  for  the  coordination  of  JFS  documents. 
In  most  cases,  the  coordination  chain  for  the  JTF 
will  be  similar  to  the  one  developed  by  the  JFS. 
The  JTF  should  verify  the  JFS  coordination 
chain  in  terms  of  currency  and  applicability  and 
establish  a  coordination  chain  for  JTF 
documents.  Development  of  the  coordination 
chain  will  allow  the  JTF  to  establish 
relationships  with  the  POCs.  The  revised 
coordination  chain  should  be  used  exclusively  for 
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JTF  ACTIVITY 
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Figure  3-2.  Example  of  JT&E  Review  Milestones 


the  coordination  of  JTF  produced  documents. 
The  established  POC  relationships  will  be  helpful 
in  obtaining  Service  resources  and  support. 

E.  REFINE  THE  APA 

The  major  components  of  the  APA  should 
be  reviewed  to  determine  if  they  are  still 
adequate.  If  not,  the  JTF  should  revise  each 
accordingly. 

Issues  and  Objectives 

The  JT&E  issues  and  objectives  developed 
during  the  JFS  were  the  foundation  of  the  APA. 
It  is  possible  that  the  JTF  coordination  chain  may 
include  organizations  not  involved  in  the  JFS  and 
these  organizations  may  insist  on  changes  or 
modifications.  Any  changes  or  modifications  to 


the  JT&E  issues  or  objectives  should  be  resolved 
by  the  JTF  as  quickly  as  possible  as  they  may 
have  a  profound  effect  on  JT&E  planning.  In 
any  event,  the  JTF  should  verify  the  currency 
and  accuracy  of  the  JT&E  issues  and  objectives 
in  the  APA  and  coordinate  any  proposed  changes 
or  revisions  with  DDT&E  (with  the  advice  of  the 
TAB).  As  the  JTF  matures,  situations  may  arise 
that  require  the  JT&E  issues  or  objectives  be 
refined.  In  such  cases,  the  changes  or  revisions 
should  be  recoordinated  with  all  agencies  and 
organizations  in  the  JTF  coordination  chain  and 
approved  by  DDT&E. 

Scenario 

The  scenario  in  the  APA  should  be  reviewed 
and  validated  by  the  operational  and  intelligence 
communities.  If  an  existing  and  validated 
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scenario  was  used  (one  that  conforms  to  defense 
planning  guidance),  the  JTF  should  review  the 
scenario  to  assure  it  is  still  valid  in  terms  of  the 
program  issues  and  objectives.  If  a  validated 
scenario  was  modified,  the  JTF  should  initiate 
the  required  actions  to  obtain  DIA  approval  of  the 
revised  scenario.  In  the  unlikely  event  that  a 
unique  scenario  was  proposed  in  the  APA,  the 
JTF  review  should  reconsider  the  use  of  that 
scenario.  If  there  is  no  alternative,  the  JTF 
should  initiate  the  required  actions  to  obtain  DIA 
approval  of  the  unique  scenario  and  should  be 
prepared  to  modify  the  scenario  and  related  test 
activities  as  necessary  to  obtain  DIA  approval. 

JT&E  Activity  Coordination 

The  JFS  program  schedule  was  based  on 
coordination  with  OSD,  Joint  Staff,  the  CINCs, 
and  the  Services  relative  to  the  potential  use  of 
test  ranges  or  facilities  or  participation  in 
scheduled  exercises.  The  JTF  must  validate  and 
formalize  the  requirement  for  use  of  these 
resources  or  participation  in  these  exercises  with 
the  facility  or  exercise  manager.  Frequently, 
coordination  in  terms  of  potential  use  of  a 
facility  or  resource  will  differ  from  coordination 
for  the  actual  use  of  a  facility  or  resource. 
Annex  G  contains  information  relative  to  the  use 
of  test  ranges  and  facilities. 

The  coordination  for  use  of  a  range  or  test 
facility  should  be  in  the  form  of  a  Program 
Initiation  Document  (PID)  that  officially 
introduces  the  JT&E  program  to  the  supporting 
range  or  test  facility.  Although  range  and  facility 
requirements  are  included  in  the  consolidated 
resource  estimate  (CRE)  and  were  coordinated  by 
Service  representatives  during  the  JT&E 


chartering  process,  the  range  or  facility  managers 
will  take  no  action  on  these  requirements  until  a 
PID  is  received.  At  this  stage  of  JTF  planning,  a 
preliminary  PID  should  be  submitted  to  ensure 
range  or  facility  availability.  PIDs  are  completed 
using  Universal  Documentation  System  (UDS) 
Document  501-79  and  the  supporting  agency 
instructions.  The  range  or  facility  manager  will 
provide  the  JTF  with  a  statement  of  capability  as 
acceptance  of  responsibilities  that  have  been 
outlined  in  the  PID. 

The  coordination  for  participation  in 
scheduled  exercises  must  be  coordinated  directly 
with  the  organization  or  agency  responsible  for 
planning  and  conducting  the  exercise.  The  JTF 
should  remember  that  exercises  are  planned  to 
achieve  specific  goals  and  JTF  participation  will 
generally  be  on  a  non-interference  basis. 
Coordination  for  participation  will  involve  in- 
depth  discussions  relative  to  potential  impacts  on 
the  exercise,  JTF  instrumentation  interfaces  with 
exercise  elements,  data  collection  and  distribution 
requirements,  control  of  JT&E  participants, 
transportation  and  logistics  support,  fiscal 
responsibilities,  and  the  control/release  of  JT&E 
results.  This  level  of  coordination  will  require 
that  the  JTF  enter  these  discussions  with  a 
complete  and  well  thought  out  plan  for 
accomplishing  the  JT&E  activity. 

Further,  the  JTD  should  be  aware  that  past 
experience  indicates  that  range  and  exercise 
managers  will  expect  the  JTF  to  provide 
something  of  benefit  to  the  range  or  exercise  in 
terms  of  cost  sharing,  instrumentation 
modifications  or  the  provision  of  equipment, 
threats,  and  instrumentation  capabilities  that 
would  not  be  available  otherwise. 
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Test  Methods  and  Test  Cell  Matrices 

JTF  coordination  for  the  use  of  resources 
(facilities  or  exercises)  to  conduct  the  activities  in 
the  APA  may  identify  the  fact  that  some  may  not 
be  available  when  needed  or  some  may  be  too 
costly.  If  either  is  the  case,  the  JTF  should 
quickly  consider  alternatives  and  formalize  the 
use  of  alternative  test  methods,  facilities  or 
resources  (exercises).  The  JFS  considered 
alternative  test  methods,  in  terms  of  cost  versus 
risk,  for  accomplishing  the  JT«&E  in  the  event  of 
reduced  program  funding  levels.  If  JT&E 
restructuring  is  required,  the  JTF  will  find  the 
JFS  test  methods  matrix  helpful  in  the 
consideration  of  alternative  activities  to  replace 
those  in  the  APA.  If  acceptable  alternatives 
cannot  be  arranged,  the  JTF  may  have  to 
consider  modifications  to  the  test  cell  matrix  or 
re-scope  the  JT&E  to  something  achievable  with 
available  facilities  or  resources.  This  rescoping 
should  include  reviews  of  the  controlled  variables 
(environments,  systems,  tasks,  etc.)  in  the  test 
cell  matrix  and  may  involve  tradeoffs  between 
facilities  and  support  resources  and  the  benefits 
to  be  gained  from  the  JT&E.  For  example,  a 
reduced  JT&E  may  be  able  to  resolve  issues  for 
only  a  limited  set  of  circumstances.  These 
limited  circumstances  may,  however,  be  those 
that  are  critical  or  those  that  will  occur  most 
frequently.  Major  changes  will  be  coordinated 
with  DDT&E. 

JT&E  Schedule 

The  APA  contains  a  schedule  that  identifies 
where  and  when  the  JT&E  activities  should  be 
conducted.  This  schedule  is  based  on 
preliminary  coordination  with  the  organizations 
responsible  for  the  resources  that  the  JTF  intends 


to  use.  The  JT&E  schedule  should  be  revised 
based  on  the  results  of  formal  agreements 
between  the  JTF  and  the  managers  of  the 
required  facilities  or  exercises.  Annex  G 
contains  a  list  of  service  test  range/facility 
documents  and  information  relative  to  the 
formalization  of  use  agreements.  Figure  3-3  is 
an  example  of  a  JT&E  schedule  that  has  been 
revised  to  accurately  reflect  when  and  where 
the  activities  and  events  will  take  place.  The 
JT&E  schedule  will  be  used  primarily  as  the 
timeline  reference  for  planning  those  actions  that 
must  be  accomplished  to  support  the  JT&E 
activities.  Such  actions  may  include  training, 
equipment  and  instrumentation  acquisitions  and 
installations,  the  transport  of  personnel  and 
materials,  modification  or  acquisition  of 
communications  or  data  collection  systems,  and 
the  provision  of  logistics  to  support  the  JT&E 
activities. 

Test  Measures 

The  JTF  should  carefully  review  each 
measure  in  the  APA  and  develop  detailed 
definitions  of  each,  to  include  explanations  of 
how  each  is  calculated.  These  definitions  should 
be  tailored  to  the  specific  test  activities  in  the 
revised  program  schedule. 

Some  of  the  measures  in  the  APA  may  be 
very  simple.  Others  may  involve  several 
computations  using  numerous  data  sources,  such 
as  determining  the  probability  of  kill  for  a 
simulated  weapons  release  against  a  fixed  facility 
target.  This  measure  requires  determining  the 
location,  velocity,  and  approach  profile  of  the 
attack  aircraft  at  the  time  of  the  simulated  release; 
wind  direction  and  speed;  several  weapon  system 
parameters;  projected  impact  point;  the 
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Figure  3-3.  Example  of  a  Revised  JT&E  Program  Schedule 


destructive  capabilities  of  the  weapon;  and  the 
size  and  construction  of  the  target.  These 
parameters  then  must  be  used  in  a  model,  such  as 
JMEM,  to  calculate  a  level  of  damage  to  the 
target.  All  of  these  parameters  result  in  data 
elements  required  to  support  the  measure  and 
must  be  included  in  the  detailed  definition  of  the 
measure  and  the  description  of  how  the  measure 
will  be  calculated. 

There  may  be  additional  data  elements 
involved  in  the  measure  definition  and 
description.  For  example,  weather  conditions 
could  affect  test  operations  and,  in  turn,  the  data 
collected.  Other  examples  include  such  things  as 


variations  in  tactics,  terrain,  or  personnel 
experience  levels  that  could  provide  variations  or 
anomalies  in  the  data  collected. 

Consolidated  Resource  Estimate  (CRE) 

The  Army  and  Air  Force  have  established 
formal  processes  to  provide  resources  to  support 
JT&Es.  The  Navy  and  Marines  have  established 
staff  responsibilities  to  provide  JT&E  support  on 
a  case-by-case  basis.  Information  on  the  Service 
processes  and  Service  directives  for  requesting 
resources  is  contained  in  Chapter  4.  These 
processes  can  take  as  much  as'  a  year  for  the 
Services  to  plan  and  provide  the  resources,  but 
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request  for  Service  resources  cannot  be  initiated 
until  after  the  JTF  is  chartered.  Request  for 
Service  resources  should,  therefore,  be 
submitted  as  soon  as  possible  after  the 
JTF  is  chartered. 

The  JFS,  in  coordination  with  the  lead  and 
participating  Services,  developed  a  CRE  that 
would  be  required  to  execute  the  JT&E.  The  JFS 
Service  deputies  should  have  broken  out  their 
portion  of  this  estimate  in  their  Service  request 
formats  and  these  breakouts  should  be  provided 
to  the  JTF  at  charter.  The  JTF  should  submit 
these  breakouts  to  the  Service  OTAs  for  the  initial 
establishment  of  JT&E  resource  requirements  in 
Service  channels.  The  JTF  should  then  revise 
these  resource  requirements  based  on  the  results 
of  the  re-coordination  of  JT&E  activities.  If  this 
re-coordination  results  in  significant  changes,  the 
JTF  should  submit  resource  requirement 
revisions  to  the  Service  OTAs  as  they  occur.  The 
JTD  should  also  establish  the  requirement  to 
review  and  revalidate  Service  resource 
requirements  at  least  annually.  All  changes  in 
JT&E  resource  requirements  should  be 
coordinated  with  the  Service  OTA  JT&E  POCs 
as  they  occur. 

F .  DOCUMENT  JTF  PLANNING 

The  APA,  as  developed  by  the  JFS  and 
approved  by  DDT&E,  focuses  primarily  on 
what  the  JTF  should  do  to  address  the  JT&E 
issues.  JTF  planning  should  focus  on  how  to 
accomplish  these  activities  and  how  the  JT&E 
results  are  to  be  documented  and  reported.  The 
JTF  should  document  this  detailed  planning.  The 
most  common  way  to  document  JTF  planning  is 
to  expand  the  level  of  detail  in  the  APA  into  a 
PTP.  Other  options  are  discussed  in  the  lead  in 


to  Part  One  of  this  Chapter.  That  portion  of  JTF 
planning  that  deals  with  how  to  collect,  manage, 
and  analyze  the  data  collected  during  the  JT&E 
activities  should  be  documented  in  a  Data 
Management  and  Analysis  Plan  (DMAP).  The 
DMAP  may  be  prepared  as  a  stand-alone 
document  or  as  an  annex  to  a  PTP.  Together, 
they  serve  as  the  JT&E  road  map  to  assure  that 
all  test  activities  are  focused  on  resolving  the 
program  issues. 

A  PTP  serves  several  purposes.  Externally, 
it  will  provide  DDT&E,  the  CINCs,  Joint  Staff 
and  the  Services  with  a  detailed  understanding  of 
the  JT&E  scope,  schedule,  milestones,  support 
requirements,  and  how  the  JTF  plans  to  resolve 
the  program  issues.  Internally,  a  PTP  will 
provide  for  the  scheduling,  coordination,  and 
acquisition  of  facilities  and  resources  to 
accomplish  the  JT&E  activities.  The  PTP  should 
specify  how  the  activities  will  be  conducted, 
when  and  where  they  will  occur,  what  units 
will  participate,  the  scenario,  the  method  of 
accomplishment,  key  data  requirements  and 
collection  techniques,  the  major  resources 
required  from  the  participating  units,  the 
composition  of  the  test  activity  teams  (during 
planning,  execution,  analysis,  and  reporting), 
and  how  the  JT&E  activities  and  events  fit  into 
answering  the  program  issues. 

The  JTF  will  refine  and  expand  on  the 
planning  actions  initiated  in  the  APA.  The  JTF 
should  then  incorporate  this  expanded  planning 
into  a  PTP  that  will  provide  the  overall 
framework  for  the  development  of  specific  plans 
for  each  of  the  JT&E  activities.  These  specific 
plans  should  outline  in  detail  where,  when, 
and  how  each  of  the  activities  will  be 
accomplished.  Annex  L  contains  an  example  of 


3-8 


JT&E  Handbook 
February  1,  1996 


an  outline  for  a  PTP  that  can  be  tailored  to  the 
requirements  of  the  JT&E. 

If  the  JT&E  is  large  or  complicated,  other 
program-level  plans  might  be  required,  such  as 
security  and  safety  plans.  These  plans  can  be 
incorporated  as  annexes  to  a  PTP  or  developed  as 
stand-alone  documents.  JTF  planning  should 
assure  that  each  of  the  site  specific  test  plans  are 
focused  on  providing  specific  inputs  to  the 
program  and  how  the  aggregation  of  the  test 
activity  results  will  be  used  to  answer  the  JT&E 
issues.  It  is  not  necessary  to  replicate 
information  in  the  APA  that  is  considered 
sufficient  to  satisfy  program  level  requirements. 
The  principal  features  of  a  PTP  are: 

•  The  finalized  JT&E  concept,  issues,  and 
objectives. 

•  Detailed  definitions  of  the  measures  and 
data  elements. 

•  Details  of  the  scenario. 

•  A  list  of  JT&E  assumptions  and 
constraints. 

•  The  matrix  that  identifies  the  test  method 
and  vehicle  for  each  JT&E  activity. 

•  The  test  cell  matrix  that  identifies  the 
controlled  variables  for  each  JT&E  event. 

•  An  approach  for  resolving  each  of  the 
issues  and  achieving  each  of  the  objectives. 

•  A  schedule  that  identifies  where  and  when 
each  JT&E  activity  will  be  conducted. 

•  An  estimate  of  resource  requirements. 

•  JTF  management  considerations. 

•  JTF  documentation  and  reporting 
requirements  and  procedures. 

•  A  description  of  the  products  that  the  JTF 
will  produce. 


The  APA  addressed  several  other 
considerations  in  the  determination  of  JT&E 
feasibility  that  should  be  reviewed  during  JT&E 
planning. 

Environmental  Considerations 

Government  regulations  mandate  that  DoD 
activities  comply  with  all  legal  statutes,  such  as 
the  National  Environment  Policy  Act  (NEPA),  to 
ensure  the  preservation  of  natural  resources.  JTF 
planning  must  consider  environmental 
requirements  in  order  to  anticipate  and  avoid 
environmental  impacts  when  possible.  For 
assistance  on  environmental  issues,  the  JTD 
should  contact  the  Service/DESA  JT&E  POC. 
Annex  I  contains  a  discussion  of  environmental 
compliance  requirements. 

Security  Considerations 

JTF  planning  must  consider  security 
requirements  in  order  to  anticipate  and  avoid 
potential  security  compromise  situations.  These 
considerations  should  include  all  aspects  of 
JT&E  operations  to  include  communications;  the 
collection,  recording,  transportation,  storage, 
reproduction,  and  destruction  of  classified  data; 
the  protection  of  classified  documents  and 
security  requirements  for  discussions;  and 
information  releases,  briefings,  and  JTF 
produced  documents.  JTF  planning  should 
address  program-level  security  procedures  and 
requirements. 

Other  Considerations 

There  are  a  number  of  other  factors  over 
which  the  JTF  may  have  little  or  no  control. 


JTiScE  Handbook 
February  1,  1996 


3-9 


Figure  3-4.  Example  of  Data  Management  and  Analysis  Functions 


Some  of  these  variables  may  include  changes  in 
available  funding,  test  support  resources  that  are 
no  longer  available  (equipment,  instrumentation, 
facilities,  ranges,  personnel,  units),  and  Service 
support  (willingness  to  provide  funding, 
personnel,  and  equipment).  The  JTF  should 
remain  sensitive  to  these  variables  and  should  be 
prepared  to  adjust  the  program  as  required  to 
complete  the  JT&E  on  time.  The  JTD  must  keep 
DDT&E  apprised  of  any  changes  in  these  factors. 

G.  DEVELOP  THE  JT&E  DATA 

MANAGEMENT  AND  ANALYSIS 

PLAN  (DMAP) 

A  JT&E  DMAP  is  the  companion  document 
to  a  PTP  that  formalizes  the  portion  of  JTF 
planning  that  deals  with  how  to  collect,  manage, 
and  analyze  data  collected  during  the  JT&E 
activities.  As  the  name  implies,  the  focus  of  the 
DMAP  is  on  data  management  collection, 
reconstruction,  verification,  storage  and  retrieval; 
and  analysis  methods,  tools,  and  outputs. 
Figure  3-4  is  an  example  of  DMAP  functions 


discussed  in  this  chapter.  The  scope  and 
requirements  for  JT&E  activities  will  be 
established  during  JTF  planning.  Regardless  of 
the  size,  scope,  or  type  of  JT&E  activities,  there 
are  many  elements  that  will  be  common  to  all 
DMAPs.  Annex  L  contains  an  example  of  a 
DMAP  outline  that  can  be  tailored  to  meet  the 
specific  needs  of  the  JTF. 

As  a  general  rule,  the  JT&E  DMAP  should 
be  prepared  as  early  as  possible  and  should  be 
coordinated  with  the  JTF  analysts  and  the  activity 
test  managers.  This  will  ensure  understanding  of 
the  data  requirements,  address  required  areas  of 
coordination  in  the  data  management  and  analysis 
process,  and  identify  possible  difficulties  and 
conflicts  in  obtaining  and  processing  the  test 
data.  The  DMAP  will  undergo  additions  and 
revisions  as  the  test  planning  matures.  A  final 
version  must  be  generated  prior  to  execution  of 
the  scheduled  JT&E  activities  as  the  specific  data 
collection,  management  and  analysis  procedures 
for  each  will  require  some  time  to  develop, 
coordinate,  and  implement. 
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The  JT&E  DMAP  provides  the  framework 
that  ties  the  overall  data  management  and  analysis 
process  together  at  the  program  level.  The  JTF 
should  anticipate  that  a  similar  but  more  detailed 
DMAP  will  be  required  to  support  the  planning 
for  each  major  JT&E  activity.  The  elements 
common  to  most  DMAPs  will  be  the 
development  of  an  analysis  methodology,  the 
identification  and  consolidation  of  data 
requirements  and  sources,  and  the  development 
of  data  collection  and  management  procedures. 
These  elements  can  then  be  consolidated  into  the 
JT&E  DMAP. 

Analysis  Methodology 

The  process  for  the  development  of  an 
analysis  methodology  begins  with  a  complete 
understanding  and  the  clarification,  if  necessary, 
of  the  JT&E  concepts,  issues,  objectives, 
measures,  and  data  requirements.  No  amount  of 
analysis  can  "fix"  poorly  defined  measures  or 
data  that  were  not  collected  accurately.  The 
derivation  of  measures  and  analysis  techniques 
may  not  be  simple,  but  the  identification  of 
analysis  support  requirements  must  be  described 
in  terms  that  are  understandable  and  defensible. 
Figure  3-5  contains  examples  of  analysis 
support  products  that  have  been  used  during 
previous  JT&Es  to  support  analysis 
requirements. 

The  analysis  support  products  should  be 
identified  as  early  in  the  planning  process  as 
possible.  Examples  of  these  products  can  be 
used  to  illustrate  and  clarify  the  test  objectives. 
When  the  analysis  support  products  have  been 
identified,  the  required  data  and  data  sources  to 
produce  these  products  should  be  evident. 
Tracing  the  analysis  support  products  back  to  the 


data  requirements  is  an  excellent  way  of  ensuring 
that  all  data  elements  have  been  identified  and  that 
unnecessary  data  will  not  be  collected.  There 
will  be  a  close  correlation  between  the  analysis 
support  products  generated  during  JT&E 
activities  and  the  JTF  final  report.  The  early 
identification  and  coordination  of  analytical 
support  products  provides  several  advantages: 

•  Test  issues  and  objectives  are  clarified  early 
in  the  planning  process. 

•  Program  and  test  managers  are  involved  in 
the  design  of  analysis  products. 

•  Traceability  is  established  between  the  test 
issues,  objectives,  measures  and  data 
elements. 

•  Only  necessary  data  are  collected  for 
analysis. 

•  Analysis  support  products  are  available  to 
support  test  management  and  reporting 
requirements. 

Analysis  is  frequently  confused  with 
"number  crunching"  and  statistics,  but  these  are 
really  sub-activities  to  the  overall  analysis 
process.  Program  analysis  involves  expert 
judgment  to  determine  the  meaningfulness  of 
observed  effects  in  terms  of  operational  impact, 
to  assess  the  importance  of  trends  that  cannot  be 
statistically  verified,  and  to  provide  insights  into 
why  observed  effects  were  (or  were  not) 
demonstrated.  In  addition,  some  issues  may 
have  to  be  answered  on  the  basis  of  aggregating 
outcomes  across  several  objectives  or  measures, 
and  aggregation  policies  are  frequently  very 
difficult  to  codify.  Some  measures  will  be 
judged  to  be  more  irnportant  than  others,  and 
some  measures  will  be  less  reliable.  Patterns  of 
results  may  suggest  conclusions  that  are  not 
supported  by  a  singular  finding.  Contradictions 
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Figure  3-5.  Examples  of  Analysis  Support  Products 


and  ambiguities  in  the  data  are  inevitable.  The 
analysis  methodology  must  consider  all  of  these 
factors  and  specify  the  analysis  procedures  in 
sufficient  detail  so  that  test  execution  becomes 
primarily  a  matter  of  following  instructions. 

The  JFS  Team  developed  an  analysis 
methodology  that  includes  both  formal  and 
informal  analysis  of  the  employed  concept  and 
system  capabilities  in  terms  of  the  mission  it 
supports.  The  JTF  will  expand  this  methodology 
to  finalize  the  analysis  criteria  that  will  be  used. 
The  JTF  should  anticipate  failures  and 
deficiencies  in  test  articles  and  other  test 


resources  and  specify  alternative  analysis 
procedures  to  be  followed  and  criteria  to  be  used 
under  such  circumstances  prior  to  actual  testing. 

Analysis  typically  will  be  accomplished  in 
three  steps;  exploratory,  confirmatory,  and 
extended. 

Exploratory  analysis  refers  to  the 
methods  and  products  by  which  analysts  come  to 
understand,  characterize,  and  investigate  the  test 
data.  The  types  of  explorations  performed  are 
very  much  dependent  on  the  type  of  data 
collected  and  the  overall  analysis  approach,  but 
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generally  will  include:  determining  the  shapes 
and  scaling  assumptions  of  underlying 
distributions;  examining  measures  of  central 
tendency  and  dispersion;  determining  criteria  and 
procedures  for  handling  outliers  and  missing 
data;  testing  assumptions  of  normality, 
independence,  and  homogeneity  of  variance  (if 
appropriate);  examining  univariate  and 
multivariate  correlations  among  test  variables. 

Common  tools  and  products  for  these 
investigations  include:  simple  descriptive 
statistics  (e.g.,  percents,  means,  quartiles), 
frequency  tables  and  summaries,  graphs  of 
distributions  (e.g.,  histograms,  cumulative 
frequency  plots),  box-and-whisker  plots,  stem- 
and-leaf  plots,  and  scatter  plots.  Information  on 
the  application  and  use  of  these  tools  can  be 
obtained  from  basic  analysis  instructions  such  as: 
Applications,  Basics,  and  Computing  of 
Exploratory  Data  Analysis,  by  Paul  Velleman. 
Sometimes  the  most  important  tools  are  non- 
statistical,  such  as  overlays  of  data  points  on 
maps  or  time  plots,  or  examination  of  operational 
logs  and  communications  to  determine 
unexpected  trends  or  relationships.  The 
complexity  and  imagination  required  for  doing 
good  exploratory  analysis  sometimes  causes  it  to 
seem  more  art  than  science.  Nonetheless,  its 
importance  for  good  testing  cannot  be  overstated. 
Every  analyst  knows  horror  stories  of  statistical 
methods  blindly  applied  resulting  in  erroneous 
and  even  ludicrous  conclusions. 

Confirmatory  analysis  is  the  term 
generally  applied  to  determining  the  statistical  and 
operational  significance  and  confidence  levels 
associated  with  the  predicted  test  outcomes. 
Where  hypothesis  testing  is  appropriate,  this  step 
will  involve  the  application  of  statistical  tests  of 


differences  in  measures  of  central  tendencies 
(e.g.,  means,  medians),  variances,  or  other 
distribution  parameters.  In  other  cases,  analysis 
may  involve  simple  comparison  of  success  and 
failure  rates  or  characterizing  performance  using 
engineering  models  or  simulations.  In  some 
cases,  human  judgment  will  be  a  primary 
analysis  method,  based  on  expert  opinion  or 
surveys  of  operators.  Again,  this  step  is  so 
dependent  on  test  requirements  that  it  is  difficult 
to  specify  the  full  range  of  what  may  be  entailed. 
The  goal  of  confirmatory  analysis  is  to  translate 
test  results  into  findings  and  conclusions  that 
address  or  resolve  the  test  issues. 

Extended  analysis  refers  to  the 
evaluation  processes  that  require  information 
beyond  the  data  collected  during  test. 
Anomalous  or  unexpected  results  may  be 
contrary  to  the  current  understanding  of  the 
system  or  its  interfaces  with  other  systems. 
Results  may  be  extended  using  modeling  and 
simulation  or  other  extrapolation  methods.  In 
most  cases,  this  step  will  address  specific  aspects 
of  cause  and  effect  in  system  performance. 

Data  Requirements  and  Sources 

JTF  planning  must  identify  the  data  that 
must  be  generated  to  meet  the  requirements 
identified  in  the  analysis  methodology  and  what 
is  the  best  data  source  and  collection  method  for 
obtaining  the  required  data  elements.  The  JTF 
should  then  develop  a  list  of  the  required  data 
elements  that  identifies  the  primary  and 
secondary  sources  and  media  for  obtaining  the 
data  (what),  establishes  the  criteria  for 
collecting  the  data  (where,  when,  how),  and 
identifies  the  responsible  party  for  collecting  the 
data  (who).  These  data  requirements  should  be 
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consolidated  into  an  Integrated  Data 
Requirements  List  (IDRL)  that  is  included  as  an 
annex  to  the  DMAP.  An  example  of  an  IDRL  is 
provided  in  Figure  3-6. 

There  may  be  additional  data  elements 
required  to  support  data  verification  and  event 
reconstruction,  to  amplify  and  support  the 
primary  measures,  or  to  provide  redundancy  in 
data  should  a  primary  data  source  fail.  As  a 
general  rule,  the  data  elements  in  the  IDRL  can  be 
grouped  into  the  following  categories: 

•  Data  elements  to  support  resolution 
of  the  measures  and  produce  the 
analysis  support  products. 

•  Ancillary  data  elements  to  collect  data 
that  will  be  used  to  explain  variations  or 
anomalies  in  the  primary  data  elements. 
For  example,  weather  conditions  that  could 
affect  test  operations. 


•  Data  elements  that  identify  the  test 
conditions  and  are  required  to  connect  and 
track  data.  For  example  trial  numbers, 
form  numbers,  or  survey  respondents 
identification. 

•  Secondary  data  and  data  sources  that 
will  be  used  if  primary  sources  fail.  Each 
trial  in  a  test  might  cost  the  Government 
millions  of  dollars.  The  instrumentation  for 
these  trials  is  never  fully  functional  nor 
reliable,  thus  requiring  the  use  of  back  up 
instrumentation  system.  Capturing  critical 
data  elements  in  more  than  one  way  can 
preserve  the  analytical  value  of  the  test  data. 

Whenever  possible,  existing  instrumented 
data  sources  should  be  used  to  minimize  the  cost 
and  resources  that  will  be  required  to  accomplish 
the  data  collection  effort.  The  JTF  should 
identify  and  obtain  detailed  descriptions  and  total 
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cost  figures  for  each  of  the  potential  sources  for 
obtaining  the  data  elements.  Cost  considerations 
should  include  such  things  as  required 
instrumentation  installations  or  modifications, 
transportation,  supplies,  operations,  and 
maintenance.  The  important  thing  is  to  consider 
all  potential  data  sources  and  determine  which 
best  suit  the  JT&E  requirements.  Figure  3-7  is 
an  example  of  a  data  source  matrix  that  should  be 
included  in  the  DMAP.  If  extensive 
instrumentation  system  descriptions  or 
specialized  procedures  are  required  to  collect  the 
data,  these  should  be  included  as  an  annex  to  the 
DMAP. 


program  database.  The  procedures  should  also 
specify  the  required  parameters  for  collecting  the 
data,  the  criteria  for  determining  the  acceptability 
of  the  collected  data,  and  a  schedule  for  the  data 
collection  activities.  Some  of  the  JT«feE  activities 
will  involve  repetitive  trials  or  events  and  others 
may  involve  test  phases  that  are  dependent  on  the 
completion  of  other  test  events.  JTF  planners 
should  be  flexible  in  designing  the  schedule  to 
assure  that  it  conforms  to  the  dynamics  of  the 
JT&E  activities  or  events  and  is  in  an  appropriate 
format  that  best  represents  those  dynamics. 

Data  Collectors 


Data  Collection  Procedures  and  Schedule 

Data  collection  procedures  should  describe 
when,  where,  how,  by  whom,  and  in 
what  formatt  all  data  elements  will  be  collected, 
as  well  as  how  the  data  will  be  transferred, 
archived,  and  controlled  for  entry  into  the 


The  DMAP  should  specify  the  number  of 
people  that  will  be  required  for  the  data  collection 
effort,  their  requisite  skills  and  training,  where 
and  when  the  people  will  be  required,  and  the 
equipment  and  support  they  will  need  at  each 
location.  There  will  be  varying  degrees  of 
human  involvement  in  all  data  collection  efforts. 
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Figure  3-7.  Example  of  a  Data  Source  Matrix 
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Even  fully  automated  instrumentation  systems 
(e.g.,  TSPI,  GPS)  require  personnel  for  setup 
and  monitoring.  There  will  frequently  be  forms 
to  fill  out  relative  to  system  performance  or  to 
provide  background  data  for  the  trial.  Other  data 
collection  systems,  such  as  video  cameras  and 
tape  recorders,  may  be  used  as  a  primary  data 
collection  device  or  as  a  backup  to  other  systems. 
Some  tests  will  require  observer  logs  and  expert 
opinion  as  primary  data  sources.  JTF  planning 
should  also  consider  shift  requirements  and 
associated  overlap  times.  Since  neither  people 
nor  instruments  are  completely  reliable,  JTF 
planning  should  consider  spares  for  both. 

Data  Processing  Flow 

The  JTF  should  develop  and  include  in  the 
DMAP  a  data  processing  flow  that  provides  for 
the  orderly  and  controlled  handling  of  data 
collected  during  each  JT&E  activity.  The 
individual  activity  teams  will  collect  data  and 
conduct  quality  assurance  and  limited  data 
reduction  on  site.  When  received  at  the  JTF 
processing  facility,  the  data  will  undergo  data 
entry  and  will  be  further  reduced  in  terms  of 
data  element  extraction,  data  fusion,  and  data 
verification.  If  large  quantities  of  data  are 
required,  the  JTF  should  consider  automated  data 
entry  screens  wherein  data  elements  are 
automatically  routed  to  the  correct  fields  and 
automated  checks  are  made  of  data  types  to 
assure  that  the  data  are  within  specified  bounds. 
Data  extraction  involves  retrieving  the  data 
from  numerous  raw  data  sources  and  translating 
these  data  into  acceptable  formats.  Data  fusion 
merges  the  data  of  different  types  and  various 
sources  into  a  format  and  file  structure  that  best 
accommodates  the  data  verification  and  truth 
analysis  process.  Data  verification  involves 


correlation  and  cross-checking  the  data  for 
consistency  and  value  acceptability.  This  data 
flow  process  is  required  to  assure  that  no  data  are 
lost  during  processing  and  that  each  step  can  be 
repeated  to  address  anomalies  or  uncertainties. 
Figure  3-8  is  an  example  of  a  data  processing 
flow. 

Data  Tracking 

Each  data  source  will  have  an  associated  data 
medium  (manual  forms,  file  on  a  computer  disk, 
audio  or  video  tape,  photograph,  system  data 
recording  tapes,  etc.).  For  each  medium,  the 
DMAP  should  include  procedures  that  provide 
for  the  positive  identification  of  each  data  element 
as  it  proceeds  through  the  data  management  and 
analysis  process.  These  procedures  should 
address  the  requirement  for  serial  numbers  on 
repetitive  forms,  data  identification  that  includes 
such  things  as  the  instrumentation  sources  and 
locations,  date  and  time  of  data  collection,  and 
format.  The  use  of  computer  printed  or  stick-on 
labels  for  serial  numbering  and  laser  scanning 
should  be  considered  in  the  development  of  the 
data  collection  and  management  procedures. 

Data  Library 

The  DMAP  should  specify  the  procedures, 
personnel,  equipment,  and  facilities  that  will  be 
required  for  data  library  operations.  Normally,  a 
field  data  library  will  be  required  at  each  test 
activity  site  that  will  be  responsible  for  the  issue 
and  control  of  data  media,  logging  the 
distribution  and  receipt  of  the  media,  and 
delivering  the  original  copies  of  the  media  to  the 
JTF  data  management  center.  A  data  library  at 
the  JTF  data  management  center  will  be 
responsible  for  the  receipt,  control,  processing. 
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Figure  3-8.  Example  of  a  Data  Process  Flow 


and  storage  of  all  test  data  in  the  JT&E  database. 
Figure  3-9  is  a  typical  example  of  JT&E  library 
functions.  The  roles  and  responsibilities  for  all 
JT&E  library  functions  should  be  clearly  defined. 

Quality  Assurance  (QA) 

The  DMAP  should  include  methods  and 
procedures  to  assure  that  data  collection  for  the 
various  test  activities  meet  the  planned 
requirements  and  that  the  collected  data  are  as 
accurate  and  complete  as  possible.  These  QA 
methods  and  procedures  should  be  planned  for 
implementation  at  all  stages  in  the  data  collection 
and  management  process.  Data  media  control 
procedures  are  an  integral  part  of  QA  and  should 
be  implemented  at  several  points  in  the  process. 
For  example,  individuals  should  be  assigned  to 
check  and  monitor  data  recording  as  soon  as 


possible  after  data  collection.  If  errors  or 
deficiencies  are  discovered  during  the  data 
collection  process,  it  may  be  possible  to  correct 
the  errors  or  re-accomplish  the  trial,  whereas 
deficiencies  that  are  discovered  later  in  the 
process  may  result  in  unrecoverable  data  losses. 

Specific  QA  techniques  will  depend  on  the 
data  collection  media.  Manual  data  forms  should 
be  checked  for  completeness,  legibility,  and 
reasonableness  of  data  magnitudes  and 
sequences.  Audio  and  video  tapes  should  be 
sampled  to  assure  that  the  recording  equipment  is 
functioning  properly  and  that  the  required  data 
fields  are  being  recorded  in  the  format  and 
magnitude  expected.  Such  factors  as  time 
constraints,  personnel,  and  equipment 
requirements  will  determine  whether  these  checks 
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Figure  3-9.  Example  of  JTF  Library  Functional  Flow 


are  accomplished  at  the  data  collection  site,  the 
database  entry  station,  or  the  JTF  data  library. 

Perhaps  the  most  important  part  of  OA  is  to 
make  sure  that  the  entire  data  collection  team 
clearly  understands  the  importance  of  the  data 
being  collected.  Each  individual  must  accept 
responsibility  for  its  accuracy.  Training  and 
practice  for  data  collectors  should  be  emphasized. 
Data  collection  procedures  should  ensure  that 
problems  are  identified  early  and  corrective 
actions  are  taken. 

JT&E  Database  Design 

The  term  database  refers  to  the  media  and 
tools  by  which  data  are  stored,  retrieved,  and 
analyzed  for  use  by  the  test  team.  Typically, 


computer  systems  are  the  preferred  media, 
although  completely  manual  systems  are 
possible.  Different  types  of  data  may  be  stored 
in  different  databases  or  in  portions  of  a  single 
multimedia  database.  Data  collected  at 
geographically  separated  locations  may  be  linked 
via  a  network  or  can  be  processed  and  merged  at 
a  single  location. 

For  most  JT&Es,  an  efficient  and  effective 
computerized  database  is  crucial  to  the  analysis 
process.  An  example  of  the  steps  in  database 
development  is  presented  as  Figure  3-10.  It 
will  be  beneficial  for  the  JTD  and  appropriate 
JTF  members  to  visit  ongoing  JT&Es  and 
discuss  the  merits  and  deficiencies  of  the 
databases  currently  in  use.  This  is  likely  to  be 
the  JTF's  most  current  and  reliable  source  of 
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Figure  3-10.  Example  of  Steps  in  Data 
Base  Development 

information  as  to  what  is  required,  what  works 
well,  and  what  does  not  work. 

Hardware  requirements  are  determined  by 
a  host  of  factors  such  as  storage  requirements. 


CPU  requirements,  data  turn-around  times, 
number  of  users,  geographic  separation, 
security,  and  file  sharing. 
Figure  3-11  is  an  example  of  a  JTF  hardware 
connectivity  diagram.  Hardware  can  be  costly 
and  may  take  some  time  to  acquire.  The  JTF 
should,  therefore,  determine  these  requirements 
early  in  the  program  and  initiate  actions  to  fund 
and  acquire  the  equipment.  The  Services  and 
DESA  maintain  some  database  equipment  that 
was  purchased  and  used  by  previous  tests  and 
JT&Es.  The  JTF  should  check  with  the  Service 
and  the  DESA  JT&E  POC  to  determine  if  any  of 
this  hardware  could  be  used  to  satisfy 
requirements  and  what  purchases  might  be 
needed  to  take  advantage  of  the  latest  technology. 

Software  needs  for  most  JT&Es  will  be 
met  with  a  commercially  available  database 
management  system  (DBMS).  The  database  will 
be  used  to  provide  reduced,  processed,  and 
analysis-ready  data  that  is  capable  of  supporting 
the  analysis  process  and  developing  the  analysis 
support  products.  If  audio  or  visual  data  are 
integral  to  the  analysis,  then  a  DBMS  with 
multimedia  capabilities  should  be  used.  The  JTF 
should  conduct  a  software  requirements  review 
to  identify  the  specific  features  needed  in  the 
database  and  analysis  software.  Depending  on 
the  analysis  methodology,  there  may  be  a 
requirement  for  separate  statistical  and  graphics 
packages.  The  need  for  utility  software  to 
generate  forms  and  reports  should  also  be 
considered.  While  the  software  requirements 
will  primarily  be  based  on  what  is  needed  to  do 
the  job,  the  JTF  should  also  consider  the  cost  of 
the  software  programs,  their  ability  to  interface 
with  other  Service  and  OSD  databases  and  the 
availability  of  existing  software. 
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Figure  3-11.  Example  of  JTF  Hardware  Connectivity 


The  Services  and  DESA  maintain  database 
and  analysis  software  programs  that  were 
purchased  and  used  in  previous  tests  and  JT&Es. 
The  JTF  should  check  with  the  Service  and 
DESA  JT&E  POC  to  determine  if  this  software  is 
applicable.  If  not,  readily  available  commercial 
off-the-shelf  software  should  be  considered. 
Many  of  these  packages  are  readily  available  and 
are  easy  to  learn  and  use.  The  use  of  off-the- 
shelf  software  may  require  that  some 
modifications  to  the  software  or  some  unique 
modules  or  queries  be  developed.  For  JT&Es 
that  involve  relatively  small  amounts  of  numerical 
data,  the  use  of  a  spreadsheet  package  may  be  all 
that  is  needed. 


Training 

The  DMAP  should  also  outline  the  types  of 
training,  the  personnel  to  be  trained,  and  an 
overall  schedule  for  training  the  data  management 
and  analysis  personnel.  Detailed  lesson  plans, 
instructional  media  and  equipment,  and  logistics 
considerations  should  be  examined  and,  if 
required,  included  in  the  JT&E  DMAP  and  the 
specific  test  activity  DMAPs.  The  amount  and 
type  of  training  will  vary  among  the  test.  Some 
training  could  be  in  the  form  of  planned 
procedures,  demonstrations  of  instrumentation, 
quick-look  analysis  of  notional  data,  or  formal 
training  sessions.  If  the  training  requirements  are 
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very  large,  the  JTF  may  want  to  consider  the 
development  of  a  training  plan  that  could  be  an 
annex  to  the  FTP  or  the  DMAP. 

H.  DEVELOP  A  TEST  PLAN  FOR 

EACH  JT&E  ACTIVITY 

While  a  quality  plan  does  not  guarantee 
successful  execution  of  a  test  activity,  poor  or 
incomplete  test  plans  assure  that  the  test  team  will 
encounter  many  problems  that  will  impact  on  the 
quality  of  the  data  collected.  The  key  to 
producing  a  good  test  plan  is  detailed  planning  to 
assure  that  the  data  are  collected  under  realistic 
operational  conditions,  all  required  data  elements 
are  collected,  the  collected  data  are  accurate  and 
complete,  and  data  collection  and  management 
does  not  corrupt  the  data.  The  detailed  planning 
should  also  identify  and  provide  for  the 
acquisition  of  adequate  instrumentation  and  test 
support  resources,  the  training  of  all  participants, 
sufficient  pretest  runs  of  instrumentation  and  data 
collection  procedures,  responsive  quick-look 
analysis  of  the  data,  and  the  flexibility  to  adjust  to 
unforeseen  circumstances  or  changing  situations. 

The  JTD  should  assign  a  test  manager  (TM) 
who  is  responsible  for  the  planning  and 
execution  of  each  test  activity.  The  TM  should 
form  a  test  team  to  assist  him  or  her  in  the 
detailed  planning  of  the  test  activity.  Normally, 
the  test  team  will  be  composed  of  personnel  that 
are  matrixed  from  the  JTF  divisions.  The  test 
team  should  then  be  augmented  as  required  from 
JTF  or  surge  resources  for  execution  of  the  test 
activity.  The  following  are  some  of  the  elements 
that  should  be  considered  in  the  development  of 
specific  test  activity  plans. 


Understand  the  APA 

Each  test  activity  is  a  part  of  the  JT&E 
program.  The  test  activity  plan  is  an  extension  of 
the  APA  that  provides  the  details  of  how  the  test 
activity  will  be  conducted.  The  TM  and  his  test 
planning  team  must  fully  understand  how, their 
test  activity  relates  to  the  JT&E  and  should 
consult  with  the  program  analysts  so  that  new  or 
revised  requirements  are  considered  and 
included. 

Deflne  the  Scenario 

Scenarios  define  the  context  within  which 
the  required  data  will  be  collected.  In  most 
cases,  field  test  activities  should  be  planned  to 
collect  data  under  operationally  realistic 
conditions.  A  scenario  should  be  defined  for 
each  test  activity  that  is  representative  of  the  APA 
scenario  but  is  tailored  to  the  specific 
requirements  of  the  test  activity.  The  scenario 
should  include  geographic  locations,  descriptions 
and  laydowns  of  friendly  and  hostile  forces,  and 
a  description  of  all  planned  operations  and 
objectives  across  a  general  time  frame.  The 
complexity  and  extent  of  the  scenario  may  be 
reduced  if  some  of  the  operational  elements  do 
not  affect  the  data  to  be  collected.  Definition  of 
the  scenario  should  be  coordinated  with  the 
supporting  Services  as  the  test  activity  results 
will  be  judged  to  a  degree  by  the  credibility  of  the 
scenario. 

Verify  and  Coordinate  Resource 
Requirements 

Resource  requirements  (units,  personnel, 
and  equipment)  were  developed  for  each  of  the 
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test  activities  during  development  of  the  APA  and 
were  submitted  to  the  Service  OTAs  for  inclusion 
in  their  resource  management  planning.  The  test 
activity  planning  teams  should  review  these 
resource  requirements  to  verify  that  all  required 
resources  have  been  included.  If  not,  the  JTF 
resource  requirement  submissions  to  the  Service 
OTAs  must  be  revised  accordingly.  The  test 
activity  planning  teams  must  then  coordinate 
directly  with  the  organizations  that  will  provide 
the  resources  and  develop  MOAs  or  PIDs  to 
assure  that  the  required  resources  will  be 
available. 

Define  Instrumentation  Requirements 

Instrumentation  requirements  should  be 
addressed  early  in  the  planning  process. 
Schedules  and  budgets  can  be  affected  by  the 
time  and  cost  required  to  procure  the  required 
instrumentation  systems  and  modify  or  integrate 
them  with  existing  systems.  The  analyst  and 
engineers  on  the  TM's  planning  team  should 
review  the  requirements  for  each  data  element 
and  determine  how  the  data  can  best  be  captured 
and  recorded. 

Most  of  the  Service  test  centers  have 
instrumentation  and  data  processing  facilities 
specifically  designed  for  operational  testing. 
Training  ranges  usually  have  less 
instrumentation.  DESA  is  a  possible  source  of 
information  on  test  ranges  and  maintains  an 
inventory  of  select,  transportable  instrumentation 
that  may  be  available  to  support  specific  test 
activity  requirements.  Acquiring  unique 
instrumentation  is  the  responsibility  of  the  JTD. 
If  required,  the  JTF  must  identify  where  the 
unique  instrumentation  can  be  obtained 
(borrowed,  purchased,  or  leased),  determine  its 


cost,  budget  for  it,  and  arrange  for  its 
acquisition.  Often,  modifying  an  existing 
instrumentation  system  at  the  host  range  or 
facility  is  a  more  attractive  option  in  terms  of  cost 
and  lead-time.  Concurrences  from  the  facility 
manager  and  system  custodian  will  be  required. 
This  concurrence  is  more  likely  if  the 
modification  will  enhance  or  extend  the  range  or 
facility  system's  capabilities. 

Define  Communication  Requirements 

Test  activity  planning  should  include 
considerations  for  communications  to  support 
such  functions  as  command  and  control,  data 
management,  security,  and  safety.  All  operating 
procedures  and  data  collection  and  transfer 
processes  should  be  monitored  so  that 
deficiencies  can  be  quickly  brought  to  the 
attention  of  the  test  activity  monitors.  Each  test 
platform  and  test  site  may  require  a 
communications  capability  for  safety 
considerations.  Test  control  may  also  require 
communication  nets.  Depending  on  the 
complexity  and  physical  separation  of  the 
communications  nodes,  large  numbers  of  nets,  or 
even  repeater  stations,  may  be  required.  Some  of 
the  requirements  may  involve  secure  nets. 

Define  Logistics  Requirements 

Logistics  support  will  primarily  consist  of 
planning  and  arranging  for  services  and  facilities 
to  support  the  test  activity  at  the  facility  or  range. 
The  required  support  will  vary  depending  on  the 
size  and  complexity  of  the  test  activity. 
Regardless,  careful  attention  to  logistics  will 
avoid  costly  delays  in  test  execution  due  to  late 
arrival  of  resources.  It  is  usually  beneficial  to 
separate  logistics  support  requirements  into  two 
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categories;  material  support  and  services  support. 
Each  should  be  included  as  an  annex  to  the  test 
activity  plan. 

Material  support  includes  the  requirement 
for  supplies,  supply  accounts,  and  special 
procedures  for  acquiring  non-standard  items  that 
the  supporting  installation  will  be  responsible  for 
procuring,  stocking,  or  handling  to  support  the 
test  activity.  Requirements  for  petroleum,  oils 
and  lubricants,  gases,  and  chemicals  must  be 
considered  as  well  as  requirements  for  the 
storage  and  handling  of  hazardous  materials. 
POCs  and  locations  for  these  services  must  also 
be  determined.  If  applicable,  requirements  for 
the  storage,  handling,  and  delivery  of  munitions; 
armament  maintenance;  and  explosive  ordnance 
disposal  must  be  considered.  A  source  of 
miscellaneous  supplies  should  be  located. 

Service  support  includes  field 
maintenance,  equipment  packaging,  equipment 
transportation,  set-up,  calibration  and  repair  of 
test  and  instrumentation  equipment, 
organizational  maintenance,  shop  services, 
billeting,  identification  badges,  food  services, 
personnel  transportation,  postal  services, 
telephone  lines,  medical  support,  recreation 
services,  aircraft  control,  weather  support,  range 
or  facility  security,  range  or  facility  access 
procedures,  and  public  relations.  Requirements 
for  administrative  and  technical  support  should 
be  specified  and  arranged  for.  Any  special 
security  measures  or  courier  service  that  will  be 
required  should  be  specified  and  arranged  for. 


Define  Manpower  Requirements 

When  test  activity  planning  has  defined  the 
data  to  be  collected,  the  data  collection  points  and 
the  instrumentation  that  will  be  required  to  collect 
the  data,  the  manpower  requirements  to 
accomplish  these  tasks  can  be  defined.  Typically 
many  sources  of  manpower  could  be  used  to 
satisfy  these  requirements.  The  most  available 
source  is  the  JTF  itself.  Large  test  activities  or 
events  may  require  that  the  test  planners  consider 
other  resources  to  satisfy  some  of  these 
requirements.  Requirements  for  support  from 
the  Services  should  be  included  in  the  JTF 
resource  requirement  request  submitted  to  the 
OTAs.  Support  contractors  are  another  source  of 
personnel  that  can  be  considered  to  satisfy  surge 
requirements. 

Develop  Detailed  Test  Activity 
Procedures 

The  planners  must  develop  detailed 
procedures  to  ensure  that  each  test  activity  is 
conducted  in  a  way  that  allows  for  the  timely  and 
accurate  collection  of  all  data  elements  in  the  test 
activity  DMAP.  These  procedures  should  be  in 
sufficient  detail  that  they  answer  any  question  a 
test  operator,  data  collector,  test  controller,  or 
support  person  might  have  about  the  conduct  of 
the  test  or  the  data  collection  effort.  The 
procedures  section  will  normally  be  the  largest 
portion  of  a  test  activity  plan.  The  test  activity 
planners  should  consider  the  following  in  the 
development  of  these  procedures: 
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•  The  planners  should  develop  a  test 
schedule  that  defines  all  facets  of  the  test 
activity  on  a  daily,  or  even  hourly  basis.  All 
participants  must  know  when  they  have  to  be 
on  site,  when  they  have  to  be  operational, 
and  how  long  they  must  be  operational. 

•  The  operational  procedures  for  the  test 
activity  should  be  operationally  realistic 
and  representative  of  Service  tactics, 
techniques  and  procedures  (TTP).  Equipment 
laydowns,  test  profiles  and  flight  paths  must 
be  provided  to  ensure  that  all  units  and 
equipment  are  correctly  located  and  that  the 
equipment  modes  and  power  settings  are 
correct. 

•  Planning  should  include  detailed  timing 
considerations  for  all  units  and  equipment. 
The  credibility  of  some  of  the  collected  data 
will  be  dependent  on  the  relationship  of  the 
test  participants  and  equipment  in  terms  of 
their  locations  and  distances  from  each  other 
at  specific  times. 

•  Data  collectors  and  test  instrumentation 
operators  must  know  when  to  initiate  and 
terminate  data  collection.  Few  test 
activities  will  require  the  continuous  and 
uninterrupted  collection  of  data,  as  this  would 
result  in  the  requirement  to  manage  large 
quantities  of  excess  data.  The  procedures  for 
each  test  activity  must  ensure  that  all  required 
data  are  collected  at  prescribed  times  and  in 
the  appropriate  format. 

•  For  some  test  activities,  the  planners  should 
consider  scripting  participant  actions  and 
responses. 

•  Unless  the  test  activity  is  "piggybacking"  on 
another  test  or  exercise,  the  procedures  must 
include  command  and  control  of  the 
JT&E  test  activity.  The  nature  and  method  of 
communicating  these  command  and  control 


procedures  to  the  participants  should  be 
clearly  established  and  defined.  Brevity 
without  ambiguity  is  critical  in  the 
development  of  command  and  control 
procedures. 

•  Test  activity  planning  should  include  the 
requirement  to  brief  all  participants  on 
their  specific  roles  before  each  test  event. 
Some  of  the  test  participants  will  not  have 
been  involved  in  the  planning  process  and 
may  not  fully  understand  their  role  or  what 
each  test  activity  should  accomplish.  All 
participants  should  also  be  debriefed  after 
each  test  event  to  determine  those  aspects  of 
the  test  event  that  went  well,  those  that  did 
not,  and  how  the  procedures  could  be 
improved.  These  briefings  and  debriefings 
should  be  carefully  planned  to  maximize  their 
benefits  and  minimize  the  time  required  for 
their  conduct. 

I .  DEVELOP  A  DMAP  FOR  EACH 
TEST  ACTIVITY 

Test  activity  planning  should  develop 
complete,  detailed,  and  integrated  procedures  for 
collecting,  identifying,  storing,  and  managing  the 
data  that  will  be  collected  during  each  test 
activity.  Unless  a  test  activity  is  very  simple,  the 
planners  should  develop  a  DMAP  for  each 
activity.  The  DMAP  should  specify  the 
procedures  that  will  be  used  to  assure  that  the 
collected  data  will  be  treated  carefully  and 
responsibly  at  every  stage  of  the  collection  and 
handling  process.  These  procedures  should  also 
address  the  transport,  storage,  and  security  of  all 
recording  media.  When  possible,  backup  copies 
of  the  media  should  be  made.  All  data  should  be 
labeled,  indexed,  and  cataloged;  procedures 
should  be  developed  to  control  and  maintain 
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accountability  of  the  data.  The  following 
considerations  should  be  addressed  in  the 
development  of  test  activity  DMAPs: 

•  Define  the  data  elements  to  be  collected 
and  specify  when  and  how  they  will  be 
collected,  the  recording  media  and  format, 
and  the  criteria  for  collecting  and  accepting 
the  data. 

•  Specify  the  instrumentation  required  for 
collecting  the  data. 

•  Specify  all  forms  required  for  collecting  the 
data,  how  the  forms  will  be  obtained,  and 
who  is  responsible  for  their  development. 

•  Specify  near-real-time  data  collection 
requirements  and  the  related  data  transfer 
methods. 

•  Specify  the  procedures  for  collecting, 
labeling,  and  distributing  the  data  media. 

•  Define  responsibilities  for  all  actions  in 
the  data  collection  and  management  process. 

•  Define  the  requirements  for  training  of  the 
data  collectors  and  managers. 

•  Estimate  the  quantity  and  types  of  data  to  be 
collected  and  determine  the  recording 
media  requirements. 

•  Specify  the  details  for  data 
management.  These  requirements  will 
include  personnel,  equipment,  and  facilities. 

•  Specify  the  procedures  for  data  quality 
assurance  and  specify  where  in  the  data 
management  process  these  procedures  will  be 
implemented. 

•  Define  the  procedures  for  quick-look 
analysis  to  ascertain  that  the  collected  data  are 
complete  and  accurate,  to  document 
compliance  with  the  test  plan,  and  to  identify 
data  collection  and  management  deficiencies 
so  that  they  can  be  corrected. 


•  Define  the  procedures  for  safeguarding 
classifled  media  and  data. 

•  Define  the  procedures  for  disposition  of  all 
data. 

If  any  portion  of  the  data  collection  and 
management  system  is  to  be  automated,  test 
activity  planning  should  include  considerations 
for  software  development.  The  use  of  milestone 
reviews  should  be  considered  throughout 
software  development  efforts  to  manage  and 
correct  slips  that  might  occur.  These  following 
considerations  relative  to  software  development 
should  be  included  in  the  DMAP: 

•  Problem  definition. 

•  Detailed  software  specifications. 

•  The  length  of  the  design  or  acquisition 
period. 

•  Timing  of  the  implementation  phase. 

•  Software  test  integration  requirements. 

•  Acceptance  criteria. 

J.  ADDITIONAL  FACTORS  TO  BE 
CONSIDERED 

Coordination  of  Resources 

All  organizations  and  agencies  scheduled  to 
provide  resources  or  services  to  the  test  activity 
should  be  contacted  to  determine  the  specifics  of 
the  support  they  will  provide  and  to  verify  that 
the  resources  and  services  will  be  available  when 
required.  This  coordination  should  be 
formalized  and  documented  to  include  such 
details  as  who  is  responsible  for  transporting  the 
resources  and  who  is  responsible  for  providing 
logistics  support  for  the  resources. 
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Scheduling 

The  scheduling  of  test  activities  is  closely 
tied  to  the  coordination  process.  The  larger  a  test 
activity,  the  more  complex  the  scheduling 
process  will  become  as  one  participant's  resource 
availability  may  conflict  with  those  of  another 
participant.  Range  or  test  facility  availabilities 
may  also  present  narrow  windows  for  activity 
accomplishment  and  flexibility  should  be  built 
into  schedules  to  accommodate  the  impact  of 
weather  or  equipment  problems.  The  following 
are  some  considerations  that  planners  should 
address  in  the  development  of  workable 
schedules: 

•  Determine  which  of  the  assets  are  most 
critical  and  which  are  the  must  difficult  to 
obtain.  Consider  these  assets  first  in 
development  of  the  schedule. 

•  Determine  if  there  is  any  flexibility  in  the 
composition  of  test  events  and  determine  the 
minimum  number  of  iterations  required  for 
each  event.  Schedule  the  least  flexible  events 
first. 

•  Determine  a  test  event  order  and  schedule  the 
most  critical  first. 

•  Assure  that  resources  and  procedures  are 
available  and  ready  for  the  start  of  each  event. 

•  Consider  the  requirement  for  makeup  trials. 

Feedback  to  the  Program 

The  data  being  collected  from  the  individual 
test  activities  will  be  used  to  calculate  measures 
and  the  measures  will  in  turn  be  used  to  resolve 
program-level  issues.  Program-level  analysts 
should  be  aware  of  what  is  being  planned  for 
theindividual  activities  and  planners  should 


provide  continual  feedback  to  the  program-level 
analysis  team.  The  program-level  analyst  and 
test  activity  planners  should  remain  cognizant  of 
the  overall  program  status  and  be  prepared  to 
refine  or  expand  test  activities  to  obtain 
incomplete  or  missing  data  from  previous  test 
activities. 

Operational  Realism 

Test  planners  will  inevitably  face  the  concern 
of  maintaining  operational  realism  in  the  test 
activity  planning  process.  Data  collection, 
instrumentation,  range,  and  safety 
constraints  frequently  infringe  on 
operational  realism.  Test  planners  must 
consult  with  operational  personnel  to  verify  that 
the  deployment  and  employment  of  systems 
closely  replicates  Service  TTPs  and  develop 
procedures  to  ensure  that  the  impact  of  data 
collection  requirements  on  realism  is  kept  to  a 
minimum. 

Environmental  Protection 

Environmental  protection  considerations  are 
of  increasing  importance  to  test  planning,  even 
those  that  will  be  conducted  on  government 
ranges  and  federal  or  state  lands.  Test  planning 
should  consider  potential  environmental  impacts 
and  develop  procedures  to  minimize,  mitigate,  or 
neutralize  any  adverse  environmental  effects. 
These  considerations  may  require  environmental 
assessments,  instmmentation  to  monitor  potential 
pollution  effects,  or  discontinuation  or  re¬ 
definition  of  the  test  activity.  Annex  I  contains 
more  detailed  information  on  environmental 
compliance. 
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Safety 

The  test  activity  plan  should  designate  a 
safety  officer,  outline  his  or  her  responsibilities, 
identify  critical  safety  aspects  of  the  test  activity, 
specify  safety  procedures  to  be  followed,  and 
establish  a  plan  for  emergencies.  The  safety 
procedures  in  the  activity  plan  should  be 
coordinated  with  the  safety  officer  at  the  range  or 
facility  where  the  test  activity  will  be  conducted. 
If  at  any  time  an  unsafe  condition  is  found, 
cognizant  personnel  should  report  the  condition 
and  operations  should  cease  until  the  safety 
hazard  is  corrected  or  eliminated.  If  the  test 
activity  involves  ground  maneuver  of  forces  or 
flying  operations,  a  safety  annex  should  be 
developed  that  addresses  such  areas  as: 

•  Overall  and  specific  safety  responsibilities 
and  requirements  for  equipment  operations. 

•  Overall  and  specific  responsibilities  for  flight 
operations. 

•  Responsibilities  and  procedures  for  ordnance 
handling,  delivery,  and  disposal. 

•  Test  activity  shutdown  criteria  and  the 
procedures  for  shutdown  execution  when 
conditions  warrant. 

•  On-site  medical  support  and  medical 
evacuation  procedures. 

•  Interfaces  with  other  related  safety  programs 
and  procedures. 

Security 

Test  activity  planning  should  consider  all 
aspects  of  security  to  include  procedures  for  the 
storage  and  handling  of  classified  materials, 
restricted  area  access,  TEMPEST,  operations 
security,  and  communications  security.  The 
planners  should  consult  the  security  portion  of 


the  APA  and  ensure  that  all  program-level 
security  requirements  are  satisfied  in  the  test 
activity  plan.  The  planners  should  also  consider 
any  unique  aspects  of  the  test  activity  and 
develop  security  plans  and  procedures 
accordingly. 

Frequency  Coordination 

If  the  test  activity  will  involve  the  use  of 
special  frequencies,  the  planners  should  identify 
such  requirements  early  and  initiate  frequency 
use  authorizations  with  the  local  Federal 
Communication  Commission  (FCC) 
representatives  at  least  six  months  before  they  are 
required.  Frequency  deconfliction  procedures 
must  be  established  and  if  jamming  is  involved, 
procedures  should  be  established  to  responsively 
terminate  the  jamming.  Frequency  deconfliction 
procedures  can  be  worked  through  the  lead 
service  frequency  deconfliction  manager. 

Equipment  Calibration 

Test  activity  plans  should  specify  that  all 
instrumentation  and  equipment  in  the  test  activity 
be  calibrated  as  often  as  the  equipment 
specifications  require.  The  test  activity  plans 
should  also  identify  who  is  responsible  for  the 
calibrations  and  where  they  will  be 
accomplished. 

Site  Surveys 

Often  the  exact  position  in  time  and  space  of 
participating  units  and  equipment  in  a  test  activity 
must  be  precisely  determined.  Test  activity 
planning  should  determine  how  these 
requirements  will  be  satisfied.  For  land 
positions,  this  may  involve  simply  obtaining 
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existing  survey  data  or  surveying  the  intended 
locations.  For  air  or  sea  positions,  flight  paths  or 
vehicle  routes  must  be  accurately  recorded  using 
location  instruments  such  as  LORAN,  TSPI 
radar,  or  Global  Positioning  Satellite  (GPS) 
systems.  Geographical  terrain  data  may  be 
required  for  line-of-sight,  equipment 
orientations,  or  relative  geometry  considerations. 
Alternate  locations  should  be  considered  for  all 
equipment  in  the  event  a  particular  location 
becomes  unavailable  prior  to  or  during  a  trial. 

Go/No-go  Criteria 

No  test  activity  will  happen  as  planned. 
When  a  problem  or  deviation  occurs,  it  is 
important  that  the  TM  quickly  make  an 
assessment  to  determine  the  utility  of  continuing 
that  specific  event.  The  planners  should  develop 
criteria  that  will  aid  in  the  decision  to  postpone  or 
cancel  an  event.  Go/no-go  criteria  are  normally 
developed  in  a  table  format  that  allows  the 
manager  to  quickly  determine  if  a  particular 
condition  will  adversely  affect  the  activity.  For 
complex  events,  the  criteria  may  address  multiple 
levels  that  allow  for  continuation  of  the  event  in  a 
degraded  mode. 

Pretest  Predictions 

The  test  activity  planners  should  conduct  a 
pretest  analysis  to  evaluate  the  benefits  of 
possible  test  configurations,  dry  run  test  data 
collection  systems,  and  determine  the  optimal 
relative  positions  of  equipment  and 
instrumentation.  Techniques  to  support  this 
requirement  can  range  from  simple  models  to 
predict  workloads  (to  size  data  collection  teams) 
to  simulations  of  event  timing,  tactics,  flight 
paths,  and  attack  profiles  to  ensure  operational 


realism.  Although  abstractions  of  the  complex 
situations  they  represent,  models  are  of  their 
greatest  utility  when  used  to  support  sensitivity 
analysis  of  a  system  to  predict  changes  that  will 
occur  when  parameters  in  the  model  are  changed. 
Pretest  predictions  will  also  provide  the  test 
manager  with  a  window  of  expected  results  that 
can  be  used  to  determine  how  the  test  activity  is 
progressing. 

Rehearsals 

Complex  tests  can  yield  invalid  conclusions 
if  the  participants  do  not  follow  the  prescribed 
procedures,  the  data  collectors  do  not  thoroughly 
understand  what  they  are  supposed  to  record,  or 
equipment  malfunctions.  Test  activity  planners 
should  consider  the  requirement  for  rehearsals  to 
maximize  the  probability  that  the  test  event  will 
be  successful.  Rehearsals  should  be  planned  to 
check  all  parameters  of  the  test  and  exercise  the 
instrumentation,  the  data  collection  and 
management  process,  and  the  operational  and 
safety  procedures.  Pretest  data  can  be  collected, 
reduced,  and  analyzed  using  established 
procedures,  tools,  and  techniques.  These 
rehearsals  should  be  conducted  sufficiently  in 
advance  of  the  test  activity  so  that  corrections  or 
revisions  to  the  test  procedures  can  be 
accommodated. 

PART  TWO 
JT&E  EXECUTION 

JT&E  activity  planning  should  have  defined 
all  of  the  functions  necessary  to  safely  and 
effectively  conduct  the  test  activities.  Execution 
should  be  a  matter  of  doing  it.  This  part  of 
Chapter  3  addresses  the  roles  and  responsibilities 
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of  the  test  manager  and  discusses  those  actions 
that  will  be  associated  with  pretest  (setting  up  the 
test,  range  coordination,  equipment  preparation, 
personnel),  test  control,  data  management,  and 
posttest  requirements. 

K.  TEST  MANAGER 

The  TM  is  responsible  for  the  safe  and 
effective  execution  of  the  test  activity.  He  or  she 
must  have  a  complete  understanding  of  the 
concept  of  structured  testing,  the  forces,  systems 
and  equipment  in  the  tests,  and  the  operation  of 
the  range  or  test  facility  where  the  testing  will  be 
conducted.  The  TM  will  be  the  primary 
coordinator  between  the  JTF,  the  test 
participants,  the  instrumentation  and  systems 
operators,  and  the  range  or  test  facility.  It  will  be 
his  or  her  responsibility  to  control  the  test  to 
assure  that  the  procedures  in  the  test  plan  and 
DMAP  are  followed  and  that  realistic  and 
accurate  data  are  collected.  Conversely,  the  TM 
must  be  prepared  to  address  unforeseen 
circumstances  and  make  adjustments  in  test 
activities  or  procedures  as  may  be  required  to 
assure  that  the  test  activity  is  conducted  safely 
and  that  the  data  collected  will  satisfy  JT&E 
program  requirements. 

The  TM  need  not  be  a  technical  expert  on 
every  aspect  of  the  test  but  must  assure  that 
technical  expertise  is  available  to  him  or  her  to  fill 
that  role.  The  primary  skill  requirements  for  a 
TM  during  test  activity  execution  are  the  ability  to 
coordinate  and  get  various  groups  to  work 
together,  the  ability  to  quickly  evaluate  situations, 
and  the  ability  to  quickly  make  decisions.  TM 
responsibilities  during  test  activity  execution  will 
require  his  or  her  total  and  undivided  attention 
during  all  phases  of  the  test  execution. 


L.  PRETEST 

Pretest  preparations  actually  begin  during  the 
planning  phase  for  the  test.  After  the  test  plan 
has  been  approved  by  the  JTD,  the  TM  should 
develop  a  schedule  for  those  actions  that  must  be 
completed  prior  to  test  execution.  Pretest 
preparations  should  include  final  range  or  facility 
use  coordination,  those  actions  required  to  assure 
that  the  participants  and  equipment  are  in  place, 
and  completion  of  the  required  training.  The  time 
between  test  plan  approval  and  test  execution 
should  be  used  to  review  all  aspects  of  the  test 
(equipment,  instrumentation,  procedures,  data 
collection  and  management,  quick-look  analysis, 
and  administration)  and  implement  corrective 
actions  as  required.  The  many  aspects  of  pretest 
preparations  are  discussed  next. 

The  TM  is  responsible  for  control  of  all 
phases  of  the  test  to  include  pretest,  test,  and 
posttest.  Most  of  the  personnel,  equipment,  and 
facilities  will  be  provided  by  other  organizations 
or  agencies.  During  the  pretest  phase,  the  TM 
must  conduct  continual  coordination  with  these 
organizations  and  agencies  to  assure  that  all 
personnel  and  equipment  arrive  on  time  and  in  a 
workable  condition  and  that  all  required  training 
is  accomplished. 

Final  Range  or  Facility  Coordination 

JTF  planning  for  the  development  of  the  test 
plan  involves  coordination  with  the  requisite 
managers  relative  to  the  availability,  control, 
support,  schedule,  and  procedures  for  use  of 
their  range  or  test  facility.  This  coordination 
should  be  continued  to  finalize  the  details  for  use 
of  the  range  or  test  facility.  The  TM  should 
emphasize  the  requirement  for  personal  two-way 
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communication  between  the  test  team  and  the 
range  or  facility  personnel. 

Equipment  Preparation 

A  key  factor  in  a  successful  test  is  that  all 
participants  and  equipment  are  in  place,  all  are 
located  in  accordance  with  the  test  plan,  and  all 
equipment  is  operating  within  the  prescribed 
parameters.  The  pretest  schedule  should  include 
provisions  for  all  participants,  equipment,  and 
instrumentation  to  be  on-site  at  the  range  or  test 
facility  sufficiently  in  advance  to  allow  for 
equipment  and  instrumentation  settings  or 
installation  and  rehearsals.  Equipment  that  does 
not  operate  within  the  prescribed  parameters 
should  be  adjusted  or  replaced.  Backup 
equipment  should  also  be  run  and  calibrated. 

Personnel 

Execution  of  most  test  activities  will  involve 
a  larger  group  of  personnel  than  those  involyed 
in  the  test  planning.  All  participants  must  have  a 
full  understanding  of  their  duties  and 
responsibilities.  They  should  also  understand  the 
importance  of  their  actions  relative  to  the  test 
activity  and  the  JT&E.  The  pretest  schedule 
should  include  provisions  for  a  thorough 
orientation  for  all  personnel  that  includes  the 
where,  what,  and  why  of  the  test  activity.  If 
the  test  activity  is  conducted  in  conjunction  with 
another  test  or  exercise,  the  relationship  of  the 
two  should  be  covered.  Hands-on 
demonstrations  of  the  systems  that  will  be  used  is 
helpful  in  explaining  system  operations, 
capabilities,  and  limitations.  A  full  explanation 
of  control  procedures,  joint  interoperability 


issues,  communications,  and  safety 
considerations  should  also  be  covered.  Finally,  a 
complete  explanation  of  the  data  collection, 
management,  and  analysis  process  is  required  for 
those  involved  in  data  collection  and 
management.  Although  not  all  of  the  participants 
will  be  responsible  for  each  of  the  functional 
areas,  a  common  understanding  of  the  test 
activity  will  aid  in  the  development  of  teamwork. 

Once  the  orientation  is  complete,  training 
and  rehearsals  on  individual  systems  or  pieces  of 
equipment  should  be  considered.  Rehearsals 
should  be  planned  to  accomplish  the  following 
four  goals: 

•  Familiarize  the  participants  with  the  most 
likely  sequence  of  events  (or  the  script  if  one 
is  to  be  used)  and  how  the  test  should 
proceed. 

•  Explore  the  impacts  and  consequences  of 
alternative  actions  that  will  prepare  the 
participants  for  making  informed  decisions 
during  the  test. 

•  Detect  planning  errors  or  misunderstandings 
relative  to  how  the  test  is  to  proceed  and  how 
the  test  forms  are  to  be  used. 

•  Develop  coordination  among  the  test  elements 
and  staff. 

The  TM  should  consider  the  use  of  scripts  in 
the  conduct  of  some  of  this  training.  Scripts  will 
assure  that  the  training  is  consistent  and  covers 
all  the  points  that  should  be  conveyed  to  the 
participants.  Scripts  may  be  used  to  explore  a 
direction  that  a  test  could  take,  for  example, 
system  failures  or  deviations  from  prescribed 
procedures.  Training  for  data  collection  and 
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management  personnel  should  consider  the  use 
of  data  that  is  representative  of  realistic  types  and 
quantities  expected  during  the  test.  The  data 
distribution  and  transfer  process  should  also  be 
exercised.  The  extent  of  the  rehearsals  should  be 
driven  by  the  complexity  of  the  systems  to  be 
used  and  the  competence  of  the  participants. 

M.  TEST  EXECUTION 

Adequate  test  planning  and  pretest 
preparations  should  result  in  test  execution  being 
a  matter  of  doing  it.  This  is  seldom  the  case 
however,  and  the  TM  must  be  prepared  to 
monitor  and  control  all  aspects  of  the  test.  Daily 
debriefs  will  help  in  improving  the  success  of 
subsequent  test  activities  and  the  identification  of 
alternative  tactics  and  techniques. 

Test  Control 

During  the  test  execution  phase,  the  TM  will 
determine  when  all  factors  are  satisfactory  for 
initiation  of  a  test  event  (system  failures, 
instrumentation,  weather,  safety,  security, 
environmental  impacts,  etc.).  The  TM  will 
exercise  control  of  all  aspects  of  the  test  activity 
and  will  terminate  testing  when  necessary.  The 
TM  will  monitor  and  assess  the  progress  of  the 
test  relative  to  problems  and  the  collection  of  data 
and  should  be  prepared  to  modify  or  substitute 
test  events  to  meet  the  overall  schedule.  The 
control  staff  in  the  test  operations  center  should 
be  able  to  display  what  is  happening  to  the  TM  in 
near-real-time.  Constant  attention  to  the  status  of 
communications  systems  and  control  procedures 
is  essential  when  assessing  the  test  situation  and 
responding  to  the  dynamics  of  the  test. 


Data  Collection  and  Management 

The  TM  is  responsible  for  ensuring  that  the 
data  collection  and  processing  activities  are 
carried  out  in  accordance  with  the  procedures  in 
the  DMAP.  Data  collection  and  reduction 
rehearsals  should  be  considered  to  reduce  the 
time  required  for  data  reduction  and  analysis. 
The  data  management  advisor  to  the  TM  must 
track  the  flow  of  the  test  to  ensure  that  deviations 
from  the  test  plan  do  not  adversely  impact  on  the 
data  collection,  management,  and  analysis 
process.  For  example,  if  statistical  analysis  is 
planned,  and  changes  result  in  insufficient 
numbers  of  samples,  the  TM  must  initiate 
corrective  actions  such  as  modifying  or  extending 
subsequent  tests.  The  data  management  advisor 
to  the  TM  is  responsible  for  quick-look  analysis 
and  related  reports.  Quick-look  is  particularly 
important  during  multi-day  tests,  where  the 
report  can  contribute  to  decisions  to  change  or 
revise  subsequent  days'  testing.  The  TM  is 
responsible  for  assuring  that  the  data  validation 
and  quality  assurance  processes  are  properly 
performed  and  that  security  procedures  are 
strictly  followed.  The  responsibility  for  lost  or 
corrupted  data  ultimately  falls  to  the  TM. 

Safety 

The  TM  is  responsible  for  the  safe  execution 
of  the  test  activity.  The  TM  will  be  assisted  in 
this  responsibility  by  a  safety  officer  who  is 
responsible  for  interfacing  with  the  host  range  or 
facility  safety  officers  on  all  safety  matters.  The 
safety  officer  will  continually  monitor  all  aspects 
of  the  test  to  identify,  report,  and  correct  safety 
deficiencies. 
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Security 

The  TM  is  responsible  for  safeguarding  all 
classified  equipment  and  all  classified  data  and 
data  media  used  or  produced  during  the  test.  The 
TM  will  be  assisted  in  this  responsibility  by  a 
security  officer  who  will  be  responsible  for 
interfacing  with  the  host  range  or  facility  security 
officers  on  all  security  matters.  The  test  security 
officer  will  continually  monitor  all  aspects  of  the 
test  to  identify,  report,  and  correct  security 
deficiencies. 

N.  POSTTEST 

During  posttest,  the  TM  is  responsible  to  see 
that  all  participants  and  equipment  are  safely 
returned  to  their  parent  units  and  that  the 
condition  of  the  equipment  is  comparable  to  the 
received  condition.  The  TM  is  responsible  for 
the  accountability  and  return  of  all  range  or 
facility  equipment  assigned  to  or  used  during  the 
test  and  the  close  out  of  all  support  arrangements 
and  agreements. 

PART  THREE 

REVIEWS,  BRIEFINGS  &  REPORTS 

The  JTD  is  responsible  for  conducting 
reviews  and  producing  briefings  and  reports  that 
will  keep  DDT&E,  Joint  Staff,  and  the 
participating  Services  apprised  of  the  JT&E 
status  and  problems  that  might  impact  on  the 
program.  The  previously  discussed  Figure  3-2 
contains  recommended  formal  program  and 
technical  reviews  at  key  milestones  in  the 
program.  The  JTD  should  consider  other 
reviews  with  agencies  and  individuals  in  the 
JT&E  process  to  stimulate  ideas  that  can  improve 
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the  program,  establish  contacts  with  resource 
providers,  and  generate  support  for 
implementation  of  the  JTF  products  and 
recommendations.  Figure  3-12  outlines  the 
basic  requirement  for  JTF  reports  and  their 
suspense. 

O.  PROGRESS  REVIEWS 

The  primary  recipient  of  JTF  progress 
reviews  will  be  DDT&E  and  the  JTD  should 
establish  a  schedule  to  provide  quarterly 
progress  briefings  to  himiher.  These 
briefings  will  review  the  financial  and  general 
status  of  the  JTF  and  present  topics  of  current 
interest  such  as  milestones,  current  issues,  status 
of  document  preparation,  and  the  results  of 
coordination.  In  addition,  a  detailed  financial 
profile,  including  commitments,  expenditure, 
remaining  balance,  and  spending  plans  should  be 
provided.  It  is  very  important  to  keep  the 
Services  informed  and  to  address  their  concerns 
through  proactive,  routine  program  reviews. 
Other  agencies  may  be  included  in  periodic  but 
less  frequent  reviews. 

In  preparing  for  these  reviews,  the  JTD 
should  develop  modular  briefings  using  a 
standard  format.  Contributors  to  the  review 
presentations  should  update  the  data  in  their 
respective  sections  at  specific  times  each  month. 
The  presentations  should  also  be  used  for  internal 
status  reviews  and  updates  to  other  agencies  as 
the  need  arises.  The  primary  thrust  of  these 
presentations,  however,  is  to  support  the 
DDT&E  quarterly  reviews;  and  inputs  to  the 
presentations  should  be  timed  and  formatted  to  be 
compatible  with  that  requirement.  The  following 
basic  elements  are  required  in  the  progress 
reviews. 
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DOCUMENTOR 

REPORT 

PRIMARY 

RESPONSIBILITY 

SUSPENSE 

DESCRIPTION  j 

§  Progress  Reports 

JTF  Chief  of  Staff 

Due  to  DDT&E  as 
required. 

Provides  periodic  updates 
of  JTF  activities.  i 

1  Financial  Reports 

Resource  Manager 

Due  to  DDT&E  as 
required. 

Reports  the  JTF  financial 
status. 

1  Program  Test  Plan 

As  established  by  JTD. 

Provides  roadmap  for  all 

JTF  activities.  ? 

\  Data  Management 
^  and  Analysis  Plan 

Data  Manager 

As  established  by  JTD. 

Orchestrates  and  guides 
data  management  efforts. 

j:  Detailed  Test  Plans 

Test  Managers 

As  established  by  JTD. 

Provides  detailed  direction 
on  test  procedures  for 
conducting  activity,  1; 

l::'Rep<)ris 

Test  Managers 

As  established  by  JTD. 

Documents  results  of  test  ^  " 

activity. 

1  special  Reports 

As  required.  NLT  45  days 
after  external  requests  if 
information  is  available- 

Reports  on  problems  that  J 

require  immediate  attention.  ^ 

j  Final  Report 

JTD 

Due  to  DDT&E  120 
days  after  completion  of  all 
test  activities. 

Final  assessment  of 
program  issue(s). 

? 

1  Management  Report 

JTD 

Due  to  DDT&E  30 
days  after  presentation  of 
final  report 

Provides  JTD  conclusions,  \ 

recommendations,  and  1 

lessons  learned. 

Figure  3-12.  JTF  Reports 


Joint  Test  and  Evaluation  Progress 

This  area  of  the  review  includes  milestone 
completions,  projections,  slippage  in  the  JT«fcE, 
problems,  challenges,  and  risks.  Items  of 
particular  importance  are  completed  and  projected 
activities,  decision  points,  and  interfaces  with 
other  agencies.  The  identification  of  long  lead 
preparations  and  projected  procurement 
requirements  should  also  be  included. 

Budget 

A  graphic  depiction  of  the  status  of 
institutional  monies  that  displays  the  committed, 
uncommitted  and  expended  status  should  be 
presented.  The  planned  and  actual  cost  of  day- 
to-day  activities  and  future  JT&E  activities 
should  be  presented  when  significant.  Direct  site 
monies,  reimbursable  funds,  and  contracts 
should  be  addressed  when  applicable. 


JTF  Problems  and  Solutions 

This  area  of  the  review  should  remain 
flexible,  as  it  is  situation  dependent. 
Nevertheless,  it  is  important  to  keep  DDT«feE 
informed  of  problem  areas  encountered, 
solutions  worked,  outstanding  problems, 
assistance  needed,  and  the  potential  impact  of  the 
problems  on  the  JT&E.  Applicable  areas  in 
which  DDT&E  awareness  is  necessary  include 
coordination  of  JTF  requirements  and  projected 
JT&E  requirements  within  OSD,  the  releasability 
of  classified  information,  resource  priorities,  and 
OSD/Service  coordination  relative  to  participation 
in  exercises,  tests,  or  the  support  of  dedicated 
JT&E  activities.  DDT&E  must  approve  any 
press  statements  or  releases. 
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P.  TECHNICAL  REVIEWS 

Technical  reviews  are  conducted  primarily 
by  the  TAB.  The  composition,  roles,  and 
procedures  for  establishment  and  use  of  the  TAB 
are  contained  in  the  JT&E  Procedures  Manual 
DoD  50003-M-4.  These  technical  reviews  serve 
two  primary  purposes.  First,  they  validate  JTF 
ideas,  approaches,  and  methodologies  using 
senior  personnel  with  unique  technical  expertise. 
Second,  they  provide  senior-level  Service 
insights  into  the  JT&E  and  can  provide  the  JTD 
with  valuable  information  and  ideas  on  how  to 
improve  the  program. 

If  the  JT&E  involves  issues  that  pertain  to 
policy,  doctrine,  or  tactics,  the  technical  reviews 
should  also  include  presentations  to  a  GOSC. 
The  GOSC  would  consist  of  senior  Service 
officers  (1-2  star  level)  that  would  advise  the  JTD 
relative  to  Service  policies,  doctrine,  tactics, 
roles,  and  missions.  The  organization  and  roles 
of  the  GOSC  members  would  be  dictated  by  the 
particular  requirements  of  the  JT&E. 

Q.  REPORTS 

It  is  important  that  the  JTD  keep  DDT&E 
informed  of  the  program  status.  Problem  areas 
encountered,  solutions  worked,  outstanding 
problems  and  assistance  required.  The  following 
reports  are  designed  to  achieve  this  objective. 

Program  Status  Reports 

In  addition  to  the  progress  and  technical 
reviews,  the  JTD  will  provide  a  monthly  program 
status/activity  report  to  the  DDT&E.  These 
reports  are  not  intended  to  duplicate  the 
information  in  the  progress  reviews,  but  rather 


keep  OSD  informed  about  how  the  JTF  is  faring, 
what  the  problems  are,  where  OSD  and  Service 
help  is  needed,  and  what  are  the  JTD 
recommended  courses  of  action.  The  JTD 
should  ask  himself  what  would  he  want  to  know 
to  help  manage  the  JT&E  and  report  that 
information.  Because  of  the  environment  in 
which  we  work,  the  report  must  also  address  the 
current  financial  status.  Funds  should  be 
delineated  by  source  and  year,  amount  received, 
committed,  obligated,  expended  and  the 
remaining  available  balance.  The  reports  should 
not  be  extremely  detailed  nor  time  consuming  for 
either  the  writer  or  the  reader.  Items  to  be 
considered  for  inclusion  are: 

•  Numbers  of  personnel  assigned.  Gains  and 
losses  in  terms  of  actual  and  projected. 

•  Trips  by  JTF  personnel  during  the  reporting 
period  and  projected  trips  for  the  upcoming 
period. 

•  Significant  visits  to  the  JTF  during  the 
reporting  period  and  projected  visits  during 
the  upcoming  period. 

•  Schedule  of  upcoming  activities. 

•  Highlighted  issues  related  to  Service 
coordination,  resources,  etc. 

•  Accomplishments. 

•  Problems. 

These  reports  are  intended  to  keep  DDT&E 
apprised  of  JTF  status  and  activities,  and  help 
DDT&E  in  his  discussions  when  contacting  or 
contacted  by  persons  or  agencies  throughout  the 
T&E  community. 

Financial  Reports 

The  JTD  will  submit  financial  reports  as 
required  or  requested  to  DDT&E  to  document  the 
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financial  status  of  the  JTF.  These  reports  will 
address  major  expenditures  and  commitments  of 
OSD  funds  and  the  status  of  the  JTF  fiscal 
situation  relative  to  fiscal  schedules  and  funds. 
The  resource  manager  is  responsible  for 
preparing  these  reports. 

JT&E  Activity  Reports 

The  JTD  will  submit  these  reports  to 
communicate  the  results  of  completed  test 
activities  (studies,  field  tests,  mini-tests, 
modeling,  etc.)  to  include  findings,  conclusions, 
and  recommendations.  Test  activity  reports  will 
include  descriptions  of  the  data  collected,  an 
analysis  of  the  data,  results  of  the  activity 
findings,  and  conclusions.  These  reports  should 
also  include  any  relevant  information  on 
instrumentation  calibration,  instrumentation 
specifications,  and  lessons  learned.  In  the  event 
of  unexpected  situations,  additional  activity 
reports  may  be  appropriate.  Collectively,  these 
reports  provide  an  audit  trail  for  the  development 
and  validation  of  the  JTF  findings,  conclusions, 
and  recommendations.  Each  report  will  address 
the  objectives  for  that  test,  and  care  will  be  taken 
to  avoid  premature  assessments  and  conclusions 
relative  to  program  issues.  JT&E  activity  reports 
may  be  provided  to  requesters  as  approved  by 
DDT&E. 

Special  Reports 

The  JTD  will  submit  special  reports  to 
DDT&E  to  surface  critical  problems  that  require 
immediate  attention  and  resolution  that  is  beyond 
his  capability.  These  reports  differ  from 
progress  reports  as  they  require  expedient 
resolution  of  a  significant  problem.  Special 
reports  may  also  be  used  to  respond  to  specific 


requests  for  data  that  may  be  generated  prior  to 
completion  of  a  test  activity  report  or  the  program 
final  report.  All  requests  for  such  data  will  be 
submitted  through  DDT&E  and  all  responses  will 
be  routed  through  DDT&E.  The  JTD  should 
exercise  caution  to  ensure  that  such  reports  do 
not  suggest  premature  conclusions  that  may  be 
based  on  incomplete  or  insufficient  data. 

When  required,  special  reports  will  be 
submitted  not  later  than  45  days  after  receipt  of  a 
request.  In  those  cases  where  the  requested 
information  is  not  yet  available,  the  JTD  will 
notify  the  requesting  agency  as  to  the  status  of 
the  request  and  the  anticipated  date  that  the 
information  will  be  available.  A  formal  briefing 
will  not  normally  be  developed  in  conjunction 
with  the  submission  of  special  reports. 

JT&E  Final  Report 

This  report  is  the  most  important  document 
that  the  JTF  will  produce.  It  is  the 
documentation  upon  which  OSD  and  the  Services 
will  base  their  decisions  relative  to 
implementation  of  JTF  recommendations  or 
institutionalization  of  JTF  developed  products.  It 
is  incumbent  on  the  JTD  to  devote  the  required 
time  and  resources  to  assure  that  the  final  report 
and  its  accompanying  briefings  are  quality 
products  and  that  they  contain  meaningful 
results,  conclusions,  and  recommendations. 
Further,  these  reports  and  briefings  must  be 
supported  by  precise  and  reasonable 
documentation.  The  JT&E  final  report  should  be 
fully  coordinated  with  OSD,  the  Joint  Staff,  and 
the  Services  before  presentation  to  the  TAB  and 
SAC.  The  final  report  should  discuss  in  detail 
the  background  and  problem  that  generated  the 
requirement  for  a  JT&E;  the  JT&E  scope, 
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scenario,  concept,  methodologies,  assumptions 
and  limitations;  the  APA;  the  JT&E  activities  and 
events;  and  the  JTF  findings,  conclusions,  and 
recommendations.  If  the  JTF  recommends  the 
institutionalization  of  JTF  developed  products, 
the  final  report  should  also  include  specific 
recommendations  for  where  and  how  the 
institutionalization  should  be  accomplished  and 
who  will  be  responsible  for  oversight  of  the 
implementation  actions.  An  example  of  a  format 
for  the  JT&E  final  report  and  briefing  is  included 
in  Annex  L.  These  formats  should  be  tailored  to 
the  specific  requirements  of  the  JTF. 

JTF  Management  Report 

The  joint  test  management  report  should  be 
completed  within  30  days  following 
presentation  of  the  JTF  final  report  to  DDT&E. 
This  report  serves  three  purposes.  First,  it 
provides  DDT&E  with  the  JTD's  assessment  of 
the  JT&E,  to  include  accomplishment  of  the 


chartered  mission.  Second,  it  documents  lessons 
learned  for  consideration  in  the  organization  and 
management  of  future  JTFs.  Third,  it  provides 
recommendations  for  actions  to  improve  the 
efficiency  and  effectiveness  of  future  JT&Es.  In 
the  preparation  of  this  report,  the  JTD  should 
separate  those  lessons  learned  and  recommended 
actions  identified  during  previous  JTF,  from 
those  that  are  new  or  unique.  Feedback  on 
problems  and  solutions  (successful  or  not)  from 
previous  JT&Es  should  be  included.  An 
example  of  a  format  for  the  JTF  management 
report  is  included  in  Annex  L.  Documenting 
lessons  learned  has  been  one  of  the 
poorly  supported  parts  of  the  JT&E 
process.  Documenting  a  lesson  learned  is 
frequently  interpreted  as  admitting  a  mistake. 
Lessons  learned  seldom  benefit  the  author  and 
thus  frequently  receive  a  minimal  level  of  effort. 
This  report,  however,  can  be  of  significant  value 
to  the  effectiveness  of  future  JT&Es. 
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CHAPTER  4 

RESOURCE  MANAGEMENT 


A.  INTRODUCTION 

Resource  management  involves  identifying 
and  quantifying  the  resources  that  will  be 
required  to  accomplish  the  JT&E  activities,  as 
well  as  programming,  budgeting,  accounting  for, 
and  controlling  the  required  resources. 

B.  RESOURCE  PLANNING 

Most  of  the  personnel,  equipment,  and 
facilities  required  to  support  the  JT&E  will  be 
provided  by  the  Services.  The  Services  will 
include  these  JT&E  resource  requirements  in 
their  resource  management  planning  to  insure  that 
the  resources  and  funding  are  available  to  support 
the  requirement.  The  JFS,  in  coordination  with 
the  lead  and  participating  Services,  should  have 
developed  an  estimate  of  the  resources  required 
to  support  daily  JTF  operations  and  to  execute 
each  of  the  JT&E  activities  in  the  APA.  These 
estimates  of  resource  should  have  been 
consolidated  into  a  single  document,  the 
Consolidated  Resource  Estimate  (CRE).  An 
outline  for  a  CRE  is  included  in  the  JFS 
Handbook  as  Annex  I. 

The  JFS  Service  deputies  should  also  have 
consolidated  those  portions  of  the  CRE  that  are 
applicable  to  their  respective  Service  and 
prepared  them  in  the  proper  format  for 
submission  to  the  Service  OTA.  Daily  JTF 
support  requirements  should  also  be  included  in 
the  resource  requirements  submitted  to  the 
Service  OTAs  and  should  be  annotated  to  reflect 
that  they  support  daily  JTF  operations.  These 


requests  for  resources  should  be  available  to  the 
JTD  at  JTF  charter.  If  the  request  for  resources 
was  done  by  the  JFS  Team,  the  JTF  should 
review  and  submit  them  to  the  respective  OTAs 
as  soon  as  possible.  This  will  provide  an  initial 
establishment  of  JTF  resource  requirements  in 
the  Service  resource  management  planning 
systems.  The  JTF  should  then  revise  these 
resource  requests  as  JTF  planning  matures  or  as 
significant  changes  occur.  If  the  JFS  did  not 
accomplish  a  CRE  or  prepare  the  Service  request 
for  resources,  the  JTD  must  make  the 
submission  of  resource  requirements  to  the 
Service  OTAs  a  priority  action  for  the  JTF. 

Requirements  for  unique  resources  must  be 
identified  so  that  JTF  programming  actions  can 
be  initiated  to  acquire  them.  These  unique 
resources  will  primarily  involve  instrumentation, 
equipment  modifications  to  accommodate  unique 
instrumentation,  instrumentation  interfaces,  and 
essential  test  control  equipment  that  cannot  be 
obtained  from  OSD  or  Service  sources. 

C .  PROGRAMMING  FOR  THE 

REQUIRED  RESOURCES 

The  JTD  should  program  for  unique  JTF 
resource  requirements  by  including  the  cost  for 
such  requirements  in  the  JTF  budget.  The  Army 
and  Air  Force  have  organizations  to  support  and 
monitor  their  Service's  involvement  in  JTFs  and 
have  established  formal  processes  to  program, 
manage,  and  provide  resources  to  joint  test 
programs.  The  Navy  and  Marines  have 
established  staff  responsibilities  for  JT&E 
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support  on  a  case-by-case  basis  along  operational 
staff  functional  lines  of  interest.  When  JT&E 
support  is  approved,  a  Navy  or  Marine  staff 
sponsor  is  assigned  to  program  for  and  provide 
the  required  resources. 

No  matter  what  form  the  Service  JT&E 
support  agencies  take,  it  should  be  noted  that  the 
process  for  programming  and  providing  the 
required  resources  can  take  in  excess  of  a  year. 
Thus,  Service  resources  may  not  be  available  for 
the  first  12-14  months  of  a  36-month  JT&E 
program.  It  is  imperative  that  JT&E 
resource  requirements  be  submitted  by 
the  JTD  to  the  Services  in  the  proper 
formats  as  soon  as  possible  after  the  JTF 
is  chartered.  The  following  are  detailed 
explanations  of  how  the  Service  systems 
program  and  provide  the  required  resources  to 
support  JT&Es. 

United  States  Army 

The  Operational  Test  and  Evaluation 
Command  (OPTEC),  in  coordination  with  the 
Deputy  Chief  of  Staff  for  Operations  (DCSOPS), 
is  the  overall  Army  manager  and  resource 
provider  for  joint  tests.  OPTEC  has  established  a 
continuing  interdepartmental  Army  committee, 
the  Test  Schedule  and  Review  Committee 
(TSARC)  that  provides  centralized  management 
of  Army  resources  to  support  all  test  activities. 
The  formal  document  for  submitting  request  for 
Army  resources  to  the  TSARC  is  the  Outline  Test 
Plan  (OTP).  New  or  revised  OTPs  are 
forwarded  to  OPTEC  for  review  and 
coordination  prior  to  presentation  to  the  TSARC 
for  their  consideration. 


The  TSARC  meets  bi-annually  to  review 
and  approve  or  disapprove  requests  for  utilization 
of  Army  resources  in  support  of  test  activities. 
The  purpose  of  the  TSARC  is  to  maximize  the 
use  of  limited  resources  and  minimize  the  impact 
of  these  activities  on  units  and  their  readiness. 
OPTEC  publishes  a  TSARC  Handbook  that 
contains  the  guidelines  and  procedures  for 
requesting  Army  resources.  Copies  of  the 
TSARC  Handbook  can  be  obtained  through 
OPTEC  CSTE-OPM,  telephone  (703)  756-1516. 
TSARC  responsibilities  include: 

•  Reviewing  and  recommending  coordinated 
OTPs  for  inclusion  in  the  Five  Year  Test  Plan 
(FYTP). 

•  Ensuring  adherence  to  the  one  year  notification 
requirement  for  resource  commitments  or 
enforcing  compliance  with  the  exception 
policy. 

•  Reviewing  and  recommending  approval  of 
T&E  priorities. 

•  Reviewing  and  coordinating  resources  for 
support  of  T&Es. 

•  Resolving  conflicts  between  user  test 
requirements  and  other  missions. 

New  OTPs  are  submitted  in  March  or 
September  of  each  year  using  the  format 
contained  as  Appendix  A  to  the  TSARC 
Handbook.  They  should  contain  maximum 
requisite  information  on  support  requirements 
available  at  the  time  of  submission.  Once 
submitted,  OTPs  are  updated  through  the 
TSARC  process  in  March  or  September  of  each 
year  as  revised  OTPs.  OTPs  are  submitted 
for  TSARC  consideration  at  least  one 
year  before  the  first  resource  is  required 
to  allow  resource  planning  against 
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approved  OTPs.  If  the  Army  resources  are  to 
be  provided  by  the  major  command  (MAJCOM) 
or  field  operating  agency  (FOA)  responsible  for 
the  OTP,  the  OTP  may  be  submitted  one  TSARC 
cycle  (i.e.  six  months)  before  the  scheduled  test 
date.  This  will  ensure  that  the  required  resources 
are  entered  into  the  FYTP  at  least  one  TSARC 
cycle  in  order  to  provide  for  the  monitoring  of 
test  resources  and  test  status.  OTPs  that  do  not 
provide  at  least  one  year  between  TSARC 
approval  and  first  resource  commitment  will  be 
processed  as  an  exception  to  policy.  Exceptions 
to  policy  OTPs  will  be  forwarded  to  the  TSARC 
under  a  cover  letter  signed  by  a  general  officer  of 
a  TSARC  member  headquarters,  stating 
justification  for  the  requested  policy  exception. 

United  States  Air  Force 

Air  Force  Instruction  99-106,  Joint  Test  and 
Evaluation,  18  Mar  94,  provides  guidelines  and 
procedures  for  Air  Force  participation  or  support 
of  JT&E  programs  directed  by  OSD.  It  describes 
the  responsibilities,  planning,  and  execution  for 
JT&E  participants.  HQ  USAF/TE  sets  Air  Force 
policy  for  JT&Es  and  is  the  office  of  primary 
responsibility  (OPR)  for  Air  Force  participation 
in  the  JT&E  program.  The  Air  Force  Operational 
Test  and  Evaluation  Center  (AFOTEC)  assists 
HQ  USAF/TE  in  the  execution  of  the  JT&E 
program  and  is  designated  as  the  Air  Force 
manager  for  chartered  JT&Es.  All  requests  for 
JT&E  dedicated  manpower  support  are 
forwarded  to  AFOTEC  for  processing  and  HQ 
USAF/TE  validation.  AFOTEC/XRJ  is  the 
AFOTEC  JT&E  action  office  and  receives 
specific  JT&E  support  requests.  The  formal 
document  for  requesting  resources  from  the  Air 
Force  to  support  joint  tests  is  the  Test  Resource 
Plan  (TRP).  The  guidelines  and  procedures  for 


requesting  Air  Force  resources  for  a  JT&E  are 
contained  in  AFOTECI  99-101  and  AFOTEC 
Pamphlet  55-8.  Copies  of  this  instruction  and 
pamphlet  can  be  obtained  by  contacting 
AFOTEC/XRJ,  telephone  (505)  846-5339. 

AFOTEC/XRJ: 

•  Assist  in  the  preparation  of  TRPs. 

•  Monitors  Air  Force  support  of  JT&Es. 

MAJCOMs  and  FOAs  will  provide  most 
of  the  resources  and  personnel  to  support 
JT&Es.  Each  MAJCOM  and  FOA  will  maintain 
a  POC  for  JT&E  matters.  Information  relative  to 
these  points  of  contact  can  be  obtained  from 
AFOTEC/XRJ.  MAJCOMs  and  FOAs  are 
responsible  for: 

•  Supporting  AFOTEC  in  interfacing  with  the 
JT&E  directors. 

•  Providing  personnel  and  equipment  as  tasked 
in  Air  Force  test  directives. 

•  Programming  resources  against  approved 
TRPs. 

•  Evaluating  JT&E  impacts  on  operational 
capabilities. 

TRP  is  the  vehicle  for  integrating  JT&E 
resource  requirements  into  the  Air  Force  resource 
planning,  programming,  budgeting,  and 
execution  system.  The  guidelines  and 
procedures  for  preparing  and  submitting  the  TRP 
are  contained  in  AFOTEC  Pamphlet  55-8.  New 
TRPs  will  be  coordinated  with  the  appropriate 
commands  and  agencies  and  forwarded  to 
AFOTEC/RM  to  coincide  with  the  two 
established  TRP  cycles,  October  through 
December  and  April  through  June.  Out-of-cycle 
TRPs  may  be  submitted  if  major  program 
restructuring  or  other  substantial  changes  occur. 
TRPs  will  be  kept  current  to  reflect  maturing 
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JT&E  resource  requirements.  TRPs  should  be 
revised  as  soon  as  program  or  test  schedules 
change  and  should  always  reflect  the  most 
current  status  of  resource  requirements. 

United  States  Navy 

The  principal  POC  for  JT&E  is  located  in  the 
Office  of  the  Director  of  Test  and  Evaluation  and 
Technology  Requirements  (N091),  specifically  in 
the  Test  and  Evaluation  Division  (N912D2).  The 
JT&E  POC  who  provides  liaison  and  interface 
with  OSD  on  JT&E  policy  and  resources  issues 
is  N912D2.  In  contrast  to  the  Air  Force  and 
Army,  the  Navy  selects  and  delegates  authority 
and  responsibility  for  JT&Es  on  a  case-by-case 
basis. 

For  Navy  support  of  a  specific  JT&E,  N091 
will  be  responsible  to  staff  the  selection  of  an 
appropriate  sponsor  from  the  Chief  of  Naval 
Operations  (CNO)  staff  organization  and  will 
assign  a  Test  and  Evaluation  Identification 
Number  (TEIN)  and  a  CNO  priority.  The  CNO 
staff  sponsor  is  responsible  for  staffing  the 
selection  of  a  field-level  Navy  agency  to  support 
the  JT&E.  The  responsibilities  and  staff  actions 
within  the  CNO  staff  for  JT&Es  are  shared 
between  N912D2  and  the  technical  sponsor.  This 
procedure  provides  a  team  approach  where  the 
JT&E  POC  is  knowledgeable  concerning  the 
entire  spectrum  of  JT&E  activity,  the  CNO 
sponsor  provides  specific  functional  expertise, 
and  the  field  level  sponsor  supports  the  JT&E. 

There  are  no  specific  Navy  directives  relative 
to  obtaining  resources  to  support  a  joint  test. 
Resource  requirements  must  be  requested  and 
arranged  for  on  a  case-by-case  basis  through  the 
CNO  and  field  level  sponsors. 


N091  is  responsible  for  obtaining  and 
assigning  the  TEIN,  the  CNO  priority,  and 
staffing  the  selection  of  the  CNO  staff  sponsor. 

CNO  sponsor  is  responsible  for  staffing 
the  selection  of  the  field  level  sponsor  and  for 
providing  the  field  level  structure  for  supporting 
JT&Es. 

Field  level  sponsor  is  responsible  for 
staffing  and  providing  resource  requirements  for 
the  JT&E  to  include  the  request  for  manpower 
billets  and  the  requisition  of  personnel. 

The  Operational  Test  and  Evaluation 
Force  (OPTEVFOR)  is  the  independent 
operational  test  agency  for  the  Navy  and  is 
responsible  for  the  operational  test  and  evaluation 
of  weapons  systems,  ships,  aircraft,  equipments, 
procedures,  and  tactics.  OPTEVFOR  is  a 
valuable  source  of  testing  expertise  and 
assistance  to  the  JT&E. 

United  States  Marine  Corps 

Within  the  Headquarters  Marine  Corps 
(HQMC)  staff,  the  responsibilities  for  JT&E 
support  are  shared  between  several  deputies.  The 
Deputy  Chief  of  Staff  for  Requirements  and 
Programs  (DC/S  R&P)  is  responsible  to 
coordinate  Program  Objective  Memorandum 
(POM)  activities.  The  Deputy  Chief  of  Staff  for 
Manpower  and  Reserve  Affairs  (DC/S  M&RA), 
after  consultation  with  Marine  Corps  System 
Command  (MCSYSCOM)  and  the  Director 
Marine  Corps  Operational  Test  and  Evaluation 
Agency  (MCOTEA),  is  responsible  for  the 
validation  of  manpower  and  personnel 
requirements  and  the  identification  of  Service 
representatives  for  approved  JT&Es.  The  Fiscal 
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Director  of  the  Marine  Corps  (FDMC)  is 
responsible  to  coordinate  with  the  DC/S  R&P  to 
ensure  that  budgetary  and  programmatic 
decisions  support  the  JT«feE,  the  Marine  Corps 
mission,  and  the  budget. 

MCO  5000. IIB  is  the  Marine  Corps 
Order  (MCO)  governing  test  and  evaluation  to 
include  participation  in  and  support  of  JT&Es. 
This  MCO  is  being  revised.  The  FSD  should 
obtain  a  current  version  by  contacting  the 
MCSYSCOM  JT&E  POC  at  telephone  (703) 
640-5963. 

Request  for  Marine  Corps  resources  to 
support  a  JT&E  are  staffed  on  a  case-by-case 
basis.  The  principal  POC  for  a  specific  JT&E  is 
MCSYSCOM,  specifically  the  Test  and 
Evaluation  Division  (MCSYSCOM/T&E). 
MCSYSCOM/T&E  will  coordinate  requests  for 
resources  with  the  appropriate  HQMC  staff  and 
MCOTEA.  The  HOMC  staff  will  coordinate 
those  actions  necessary  to  provide  the  approved 
resource  support.  MCOTEA  will  recommend  to 
the  Commandant  of  the  Marine  Corps  (CMC)  the 
selection  of  a  Fleet  Marine  Force  (FMF)  to 
support  the  joint  testing.  The  FMF  will  designate 
FMF  coordinators  at  applicable  units  and  provide 
the  approved  resources  and  support. 

MCSYSCOM/T&E  is  the  principal  POC 
for  all  JT&E  support  requirements  to  include 
technical  assistance,  personnel,  equipment,  and 
forces. 

DC/S  R&P,  after  consultation  with 
MCSYSCOM  and  the  Director  MCOTEA,  is 
responsible  for  the  validation  of  manpower  and 
personnel  requirements  and  the  identification  of 


Service  representatives  for  approved  JT&E 
positions. 

MCOTEA  is  the  Marine  Corps  independent 
T&E  organization  responsible  for  adequate 
testing,  objective  evaluation,  and  independent 
reporting  in  support  of  the  Marine  Corps 
acquisition  process.  MCOTEA  is  also 
responsible  to  recommend  to  CMC  the  selection 
of  the  FMF  to  support  joint  testing;  to  provide 
technical  support  to  the  Marine  Corps  joint 
technical  director  in  the  form  of  assistance  in  test 
planning,  direction,  and  reporting;  to  ensure  the 
quality  control  of  Marine  Corps  participation;  and 
to  conduct  independent  evaluations  as  may  be 
required. 

FMFs  are  responsible  to  designate  test 
coordinators  in  the  FMF  headquarters  and  at  each 
division  or  wing  and  to  provide  the  approved 
resources  required  to  conduct  or  support  the 
JT&E. 

D.  RESOURCE  BUDGETING 

Budgeting  for  the  required  resources 
involves  two  functions,  estimating  the  cost  of  the 
required  resources  and  including  these  costs  in 
the  applicable  budgets.  Chapter  2  (Figure  2-1) 
outlines  the  OSD  and  Service  responsibilities  for 
providing  the  required  JTF  expenditure  and 
resources.  Once  the  resource  requirements  are 
submitted  to  the  Service  OTAs,  the  Services  are 
responsible  for  including  the  JT&E  resource 
requirements  in  their  resource  management 
planning  systems  and  including  these 
requirements  in  their  respective  budgets.  Those 
resource  requirements  that  are  an  OSD 
responsibility  should  be  included  in  the  JTF 
budget. 
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Cost  Estimates 

Cost  estimates  of  the  required  resources 
should  be  developed  by  quantifying  each  of  the 
required  resources  in  terms  of  what  will  be 
required;  where,  when,  how  long  it  will  be 
required;  and  how  it  will  be  provided  at  the 
required  location.  This  is  accomplished  by 
decomposing  the  CRE  into  specific  items, 
costing  each,  and  then  aggregating  the  cost  by 
time  and  responsibility.  The  CRE  decomposition 
should  be  tailored  to  the  unique  requirements  of 
the  JT&E.  The  following  are  suggested  topics 
that  should  be  considered  in  a  breakdown  of  the 
resource  requirements  in  the  CRE. 

•  Category  (personnel,  facility  requirements, 
transportation,  materials,  support  equipment, 
aircraft  sorties,  etc.). 

•  Type  (rank,  technical  qualifications,  type  of 
aircraft,  secure  space,  etc.). 

•  Location  (home,  deployed,  labs,  ranges, 
etc.). 

•  Duration. 

•  Relocation/transportation  requirements. 

Each  JT&E  is  unique  and  may  require 
unique  data  collection  and  analysis  equipment. 
However,  there  are  many  items  and  requirements 
that  are  similar  and  constant  between  JT&Es. 
Annex  H  is  an  example  of  a  cost/budget  matrix 
that  can  be  used  to  identify  and  cost  JT&E 
resource  requirements.  The  matrix  will  have  to 
be  adjusted  and  tailored  by  the  JTF  to  meet  the 
individual  needs  of  the  JT&E. 

The  categorization  of  resource  requirements 
will  allow  the  JTF  to  fully  define  each  in  order  to 
more  accurately  estimate  its  cost.  The  cost  for 
the  use  of  range  and  laboratory  facilities  should 


be  obtained  directly  from  the  facility  operators. 
The  cost  associated  with  Service  provided 
resources  could  be  obtained  by  the  JTF  deputies 
through  interface  with  their  Service's  operational 
commands  and  logistics  agencies.  The  cost  of 
satisfying  unique  JTF  hardware,  software, 
instrumentation,  interface,  and/or  modifications 
should  be  obtained  from  the  most  reliable  source 
available  to  the  JTD  (OTA,  labs,  ranges, 
vendors,  etc.).  DESA  may  be  a  source  of  cost 
information,  as  well  as  the  provision  of 
instrumentation,  data  collection,  and  analysis 
hardware  and  software. 

The  cost  of  the  individual  items  are  then 
aggregated  by  location,  time,  and  responsibilities 
(OSD,  the  Services)  and  incorporated  into  the 
applicable  budgets.  If  the  estimated  cost  for  the 
unique  requirements  exceeds  the  fiscal  guidelines 
outlined  for  the  JT&E  by  DDA&SP,  the  JTF 
should  consider  programmatic  or  technical 
restructuring  of  the  JT&E  to  reduce  the  estimated 
cost.  The  following  are  guidelines  for 
determining  cost  responsibilities  of  the  CRE 
items: 

OSD  provides  partial  funding  to  JT&Es  in 
accordance  with  the  DoD  Budget  Guidance 
Manual  (DoD  Manual  7110-1).  These  OSD  funds 
(PE  0605804D)  are  used  to  pay  costs  incurred 
for  the  direction,  supervision,  and  performance 
of  the  JT&E  and  those  that  are  unique  to  JT&E 
needs  such  as  the  procurement,  installation  and 
operation  of  unique  instrumentation; 
transportation,  travel,  and  per  diem  cost  for  the 
JTF  staff;  modifications  of  test  articles;  data 
collection,  reduction  and  analysis,  and  test 
reporting;  and  reimbursable  cost  identified  in 
DoD  3200.11,  Use  Management  and  Operation 
of  DoD  Ranges  and  Test  Facilities.  Thus,  costs 
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that  are  incurred  solely  as  a  result  of  conducting  a 
JT&E  and  that  cannot  be  regarded  as  providing 
long-term  mission-oriented  benefits  are  normally 
funded  by  OSD. 

Services,  through  operations  and 
maintenance  budgets,  are  responsible  for  funding 
normal  operational  expenditures  such  as  civilian 
pay,  travel  (for  other  than  the  JTF  staff), 
transportation,  utilities  and  rents.  Service  owned 
equipment  maintenance,  supplies,  printing, 
reproduction,  and  communications  in  support  of 
a  JT&E.  The  Services  also  provide  the  support 
budgets  and  funds  for  normal  operations  such  as 
the  cost  of  non-industrial  funded  aircraft 
operations,  OPTEMPO,  TDY  cost  for  Service 
exercise  participants,  and  personnel  other  than 
the  JTF  staff  who  are  involved  in  test  activities, 
and  common  support  of  Service  test  equipment. 
The  JTD  and  the  JTF  resource  manager  will 
monitor  the  status  of  funds  allocations, 
commitments,  and  utilization  and  will  provide 
reports  on  fiscal  status  and  requirements. 

E.  RESOURCE  ACCOUNTABILITY 

The  JTD  is  accountable  for  the  control,  use, 
and  expenditure  of  all  resources  assigned  to  or 
used  during  the  JT&E.  The  test  managers  are 
responsible  to  the  JTD  for  the  control,  use,  and 
expenditure  of  all  resources  used  in  support  of 
their  test  activities.  The  JTD  and  test  managers 
are  supported  by  a  JTF  resource  manager.  The 
JTF  resource  manager  is  responsible  for  tracking 
all  assigned,  acquired,  procured,  and  allocated 
resources  and  preparing  all  resource  related 
reports.  The  JTF  resource  manager  is  also 
responsible  for  keeping  the  JTD  apprised  of 


resource  situations  that  could  impact  on  the 
JT&E.  Resource  accountability  will  basically 
involve  two  functions,  funds  and  assets 
accountability. 

Funds 

The  JTD  must  coordinate  with  the 
supporting  and  finance  office  at  the  host 
installation  to  establish  an  account  and  system  to 
track  the  commitment,  obligation,  and 
expenditure  of  funds  throughout  the  life  of  the 
JTF.  The  established  account  and  tracking 
system  will  be  used  to  track  funds  and 
expenditures  and  enable  the  JTF  resource 
manager  to  identify  trends  and  cost  factors  that 
may  project  a  deficit  or  surplus  of  funds.  JT&Es 
are  partially  funded  by  OSD  in  accordance  with 
the  directions  contained  in  DoD  Manual  7110-1. 
These  funds  are  used  to  pay  costs  that  are  unique 
to  the  needs  of  the  JT&E  such  as  the 
procurement,  installation,  and  operation  of 
unique  instrumentation;  transportation,  travel, 
and  per  diem  costs  of  the  JTD  and  staff;  test 
article  modifications  to  obtain  test  data;  data 
collection,  reduction,  and  analysis;  and  test 
reporting  for  the  involved  Services.  The 
Services  are  reimbursed  for  unique  costs  incurred 
as  a  result  of  the  JT&E  but  not  for  normal  O&M 
expenditures  such  as  civilian  salaries,  travel  for 
other  than  the  JTF  staff,  transportation,  utilities 
and  rents,  Service-owned  equipment 
maintenance,  supplies,  printing  and 
reproduction,  and  communications  in  support  of 
the  JTF.  A  Military  Interdepartmental  Purchase 
Request  (MIPR)  will  normally  be  used  for  the 
procurement  of  materials.  Services,  supplies,  and 
equipment  purchased  from  other  DoD 
components. 
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Assets  Accountability 

Property  accountability  of  JTF  assets  will  be 
maintained  through  a  system  of  property  books 
and  hand  receipts.  The  property  books  should 
list  all  JTF  properties  (assigned  or  acquired)  by 
type,  serial  number,  who  the  property  is  assigned 
to,  where  the  property  is  located,  and  the 
organization  from  which  the  property  was 
obtained.  Each  property  should  be  uniquely 
marked  with  a  JTF  identification  and  serial 
number.  Accountable  property  procured  with 
DoD  funds  will  be  recorded  in  the  DESA 
property  book  as  they  are  the  custodian  for 
DDT&E  property.  The  JTD  will  receipt  for 
installation  property  received  from  the  designated 
host  installation  organization  or  officer.  Service 
resources  used  during  a  JT&E  will  remain  on  the 
property  books  of  the  organization  providing  the 
resource.  If  a  representative  from  the  providing 
organization  is  not  on-site  at  the  test  activity 
location,  the  test  manager  will  receipt  for  the 
property  to  the  owning  organization.  A  detailed 
resource  list  will  be  developed  for  each  test 
activity  and  will  be  updated  throughout  the 


planning  process  as  requirements  are  refined. 
The  detailed  resource  list  will  include  all  tactical 
and  non-tactical  equipment  such  as 
instrumentation,  communication  systems,  tools, 
and  expendable  materials.  The  JTF  should 
establish  a  local  checking  account  (imprest 
account)  with  the  host  installation  to  support 
unexpected  supply  requirements. 

Key  resources  (personnel,  aircraft,  flying 
hours,  test  range  hours,  munitions,  etc.)  used  to 
accomplish  the  JT&E  will  be  documented  in  the 
test  report  or  other  media  to  be  used  for  future 
test  resource  estimates  or  for  responding  to 
inquiries. 

The  JTD  is  responsible  for  the  disposal  of  all 
JTF  resources  and  the  test  managers  are 
responsible  for  the  disbanding  of  test  teams  and 
the  disposal  of  resources  associated  with  their 
test  activity.  The  JTF  should  develop  a  checklist 
for  assuring  that  all  acquired  resources  are 
returned  in  proper  condition  to  the  appropriate 
custodians  or  are  disposed  of  in  accordance  with 
applicable  directives. 
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CHAPTER  5 
CLOSE  DOWN  OF  JTF 


A.  INTRODUCTION 

Close  down  is  the  last  phase  of  the  JTF  and 
perhaps  the  most  demanding  in  terms  of  the  work 
to  be  accomplished  within  the  time  available. 
The  JTD  should  initiate  the  development  of  a 
simple  Close  Down  Plan  (CDP)  early  in  the  JTF 
to  identify  those  actions  that  must  be 
accomplished  and  a  schedule  for  their 
completion.  The  identified  actions  will  include 
the  preparation  and  coordination  of  the  final 
report,  the  release  of  personnel,  personnel 
efficiency  reports,  personnel  awards  and 
recognitions,  the  accounting  for  and  return  of 
property,  the  termination  of  contracts,  the 
termination  of  support  agreements,  the  close  out 
of  fiscal  accounts,  and  the  return  of  facilities. 

B  .  FINAL  REPORT  COORDINATION 

AND  BRIEFINGS 

Preparation  of  the  JTF  final  report  should 
start  early  and  should  continue  in  parallel  with 
completion  of  JT&E  activities.  Some  of  the 
report  "boiler  plate"  can  and  should  he 
developed  as  early  as  possible.  Sections  of 
the  report  should  be  completed  as  data  and 
information  are  collected  and  analyzed  and 
conclusions  are  reached  relative  to  program 
issues.  A  final  draft  of  the  report  should  be 
completed  in  sufficient  time  to  allow  for 
coordination  with  OSD,  the  Joint  Staff,  and  the 
Services  prior  to  submission  to  the  TAB  and  the 
SAC.  Coordination  should  be  in  accordance  with 


the  coordination  chain  previously  established. 
This  can  take  up  to  six  weeks  with  revisions,  and 
follow  on  coordination  may  take  another  two 
weeks.  The  JT&E  final  report  will  be  presented 
to  the  TAB  and  SAC  for  concurrence  and  to 
DDT&E  for  approval. 

Finalization,  production  and  distribution  of 
the  JT&E  final  report  should  be  completed  at 
least  two  weeks  prior  to  the  scheduled  close 
down  date.  Any  continuation  of  work  on  these 
documents  after  that  date  can  be  affected  by  the 
administrative  and  logistical  actions  associated 
with  the  close  down  of  the  JTF.  These  close 
down  actions  include  the  transfer  of  the 
remaining  assets  (computers,  printers,  ADP, 
etc.)  back  to  the  controlling  organizations  or 
agencies  and  the  transfer  of  the  JTF  facilities 
back  to  the  host  installation. 

The  JTD  should  present  briefings  on  the 
findings,  conclusions,  and  recommendations  of 
the  JT&E  to  key  DoD,  Joint  Staff,  and  Service 
organizations  in  conjunction  with  the 
coordination  of  the  JTF  final  report.  These 
briefings  are  particularly  important  for 
gaining  acceptance  and  support  of  the 
JT&E  findings,  conclusions,  products 
and  recommendations .  Conduct  of  these 
briefings  can  be  time  consuming  and  will 
necessitate  planning  and  coordination  to 
maximize  the  number  of  briefings  that  can  be 
accomplished  on  each  trip.  Read-ahead 
copies  of  the  briefings  should  be 
provided  to  the  respective  organizations  prior  to 
scheduled  briefings. 
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C .  INSTITUTIONALIZATION 

By  definition,  joint  tests  address  and 
evaluate  solutions  to  problems  or  deficiencies  in 
joint  operations.  In  most  cases,  the  JTF 
evaluation  of  the  JT&E  concepts  and  issues  will 
result  in  specific  recommendations  for  changes  in 
policies  and  procedures.  In  some  cases,  the  JTF 
will  develop  products  that  represent  significant 
improvements  in  operational  capabilities  or  the 
way  that  OSD,  the  Joint  Staff,  the  CINCs,  and 
the  Services  do  business.  JTF  recommendations 
and  products  can  come  in  many  forms  that  range 
from  changes  in  policy  directives  to 
methodologies  for  evaluating  operational 
systems.  None  of  the  JTF  recommendations  and 
products  represent  improvement  in  joint 
operation  capabilities,  however,  unless  they  are 
implemented  and  institutionalized. 

The  JTF  should  conduct  a  study  to  evaluate 
the  requirement  for  implementation  of  the  JTF 
recommendations  and  products.  The  study 
should  as  a  minimum  address  the  following: 

•  What  specifically  is  the  problem? 

•  How  large  is  the  problem? 

•  What  causes  the  problem? 

•  Identify  corrections  to  the  problem. 

•  Identify  who  should  be  responsible  for 
implementing  the  recommended  corrections. 

•  Identify  who  should  be  responsible  to 
monitor  the  corrective  actions  to  assure  that 
the  problem  does  not  recur. 

•  Determine  the  cost  to  implement  in  terms  of 
manpower  and  budget. 

•  Determine  an  estimate  of  savings  and 
improvements  in  efficiency  or  joint 
capabilities. 


If  the  JTF  study  determines  that 
implementation  actions  are  warranted,  the  JTF 
should  develop  a  plan  for  institutionalization  of 
the  results  and  products.  This  plan  should 
address  all  facets  of  the  JTF  recommendations  to 
include  who,  where,  and  how  the  JTF 
recommendations  and  products  should  be 
institutionalized.  The  plan  should  be  coordinated 
with  all  agencies  or  organizations  that  would  be 
involved  in  the  institutionalization.  The  plan 
should  then  be  presented  to  DDT&E  for 
approval. 

D.  PERSONNEL 

Once  the  final  JT&E  test  activities  are 
completed,  the  focus  of  the  JTF  and  the 
corresponding  personnel  needs  will  shift  to 
program-level  activities  such  as  analysis, 
compilation  of  information  for  the  production  of 
program  level  reports,  and  the  inventory  and 
disposition  of  assets.  These  activities  will 
involve  specific  personnel  resources,  which  in 
most  cases  will  not  include  all  personnel  assigned 
to  the  JTF.  As  such,  some  personnel  can  be 
released  after  completion  of  the  final  test  activity 
and  others  can  be  released  shortly  after 
completion  of  certain  close  down  actions. 

The  JTD  should  identify  the  personnel  who 
need  to  remain  until  close  down  and  develop  a 
time-phased  schedule  for  their  release  and 
transfer.  This  schedule  should  be  accomplished 
8-12  months  before  close  down  so  that  all 
personnel  have  an  approximate  date  for 
reassignment  (to  initiate  actions  to  obtain  another 
position)  and  so  that  all  know  what  is  expected  of 
them  after  completion  of  the  final  test  activity. 
Failure  of  the  JTD  to  take  early  action  .in  this  area 
could  result  in  personnel  initiating  their  own 


5-2 


JT&E  Handbook 
February  7,  1996 


actions  relative  to  follow-on  employment  and 
assignments,  which  could  result  in  the 
unexpected  departure  of  key  personnel  during  the 
critical  close  down  phase. 

Service  personnel  will  begin  to  focus  on 
their  next  assignment  as  they  see  the  end  of  their 
current  assignment  approaching.  Most  will 
contact  their  personnel  assignment  agencies  to 
express  desires  for  specific  assignments  and  to 
influence  their  reassignment  dates.  Many  will 
desire  to  move  during  the  summer  months. 
Personnel  being  reassigned  to  career 
development  schools  will  be  departing  in  the 
June-August  time  frame.  Given  this  situation, 
the  JTD  should  consider  both  the  JTF 
requirements  and  the  individual's  career.  There 
will  be  tough  decisions  to  make  regarding  the 
select  personnel  that  are  to  remain  until  close 
down  and  the  JTD  should  involve  the  Service 
deputies  in  these  decisions.  Once  the  decisions 
have  been  made,  the  information  should  be 
conveyed  to  the  respective  military  personnel 
centers  so  that  appropriate  reassignment  actions 
can  be  initiated.  This  should  be  done  at  least  six 
months  prior  to  completion  of  the  final  JT&E  test 
activity.  The  JTD  and  deputies  should  recognize 
that  they  may  have  little  control  over  the 
reassignment  of  some  personnel,  particularly 
those  scheduled  to  attend  Service  schools. 

Civil  Service  personnel  will  most  likely 
be  reassigned  to  organizations  on  the  host 
installation.  The  JTD  should  alert  and 
coordinate  with  the  host  installation  Civilian 
Personnel  Office  (CPO)  relative  to  the  projected 
JTF  completion  date  and  the  necessity  to  find 
jobs  for  the  Civil  Service  personnel.  The  transfer 
of  Civil  Service  personnel  is  dependent  on 
existing  vacancies,  the  career  fields  and  grade 


levels  of  these  vacancies,  and  the  interview  and 
acceptance  of  the  individuals  by  management 
personnel. 

With  few  exceptions,  the  JTD  must  be 
prepared  to  release  Civil  Service  personnel  as 
soon  as  they  are  able  to  obtain  a  position  in 
another  organization.  This  may  create  voids  in 
the  capabilities  required  to  finalize  JTF  products 
and  necessitate  that  the  JTD  consider  alternatives 
for  obtaining  the  required  support. 

Contractor  personnel  often  constitute  the 
most  stable  personnel  resource  available  to  the 
JTF  during  close  down  as  they  do  not  have  the 
same  reassignment  pressures  as  the  military  or 
Civil  Service  personnel.  Thus,  they  will  remain 
available  to  support  the  JTF  up  to  the  closing 
date.  Contractor  personnel  may  be  required  to 
fill  voids  created  by  the  early  release  of  military 
or  Civil  Service  personnel.  The  use  of  contractor 
personnel  to  provide  this  augmentation  may 
necessitate  a  revision  of  the  support  subtask  to 
assure  that  the  required  support  is  within  the 
scope  of  the  contract. 

Appraisal  Reports 

During  the  close  down  phase,  the  JTD 
should  give  special  attention  to  personnel 
appraisal  reports.  This  involves  preparation  and 
submission  of  the  appraisal  reports  or  providing 
inputs  to  the  individuals  rating  officials.  In  the 
case  of  Civil  Service  personnel,  failure  to  submit 
or  provide  a  timely  input  to  an  appraisal  report 
could  have  serious  consequences  on  the 
individual.  The  intensity  of  activity  during  the 
close  down  phase  and  the  focus  on  work 
completion  often  results  in  appraisal  inputs  being 
overlooked  or  hastily  provided.  Once  again,  the 
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CDP  should  include  a  schedule  for  providing 
inputs  into  the  personnel  appraisal  systems. 

Awards  and  Recognition 

As  the  JTF  nears  completion,  attention  needs 
to  be  given  to  the  recognition  of  those  personnel 
and  organizations  that  have  provided 
extraordinary  support  to  the  JTF.  Since  the  JTF 
is  a  joint  activity,  formal  awards  to  individuals 
for  performance  on  the  JTF  should  be  joint 
awards.  In  many  cases  assignment  to  the  JTF 
may  represent  the  individual’s  only  opportunity 
to  receive  a  joint  award.  OSD  award 
recommendations  should  be  forwarded  to 
DDT&E  90  days  prior  to  the  desired  date  of 
presentation  for  review.  The  recommendation 
will  be  forwarded  to  a  joint  awards  board  for 
consideration  and  approval.  In  some  cases,  the 
disapproval  of  a  recommendation  will  be 
conveyed  through  the  approval  of  a  lessor  award. 
If  there  is  cause  to  rebut  the  downgrade,  the  JTF 
should  submit  another  recommendation  with  a 
stronger  justification.  JTDs  should  be  prepared  to 
rebut  a  boards'  downgrading  if  there  are  strong 
feelings  about  the  individual's  performance  and 
accomplishments.  Submission  of  personnel  for  a 
Service  award  in  lieu  of  a  joint  award  is  an 
alternative.  In  no  case  should  an 
individual  be  submitted  for  both  a  joint 
and  Service  award  for  performance,  or 
achievement  associated  with  the  JTF. 
Service  recommendations  for  awards  should  be 
submitted  by  the  Service  deputies  in  accordance 
with  Service  procedures. 

Each  Service  operates  differently,  and  the 
individual  may  have  preferences  relative  to  when 
and  where  an  award  is  to  be  presented.  Both 
must  be  considered  relative  to  the  establishment 


of  a  suspense  for  award  submissions.  When  and 
where  the  award  will  be  presented  may  also  be 
influenced  by  a  downgrading  and  resubmission 
of  a  recommendation. 

Awards  and  recommendations  for  Civil 
Service  personnel  should  be  coordinated  with 
and  submitted  through  the  host  installation  CPO. 
JTFs  may  also  consider  presentations  of  awards 
to  organizations  and  individuals  that  provided 
support  to  the  JTF  or  to  individuals  who 
contributed  significantly  to  the  JTF's 
accomplishments.  Some  of  these  awards  could 
be  Joint  Achievement  Awards  to  organizations  or 
individuals  for  their  contributions  during  a 
particular  time  or  event.  OSD  Certificates  of 
Achievement  are  available  to  give  special 
recognition  to  supporting  organizations  or 
individuals.  These  two  recognitions  require 
formal  submissions  with  justification  to  DDT&E 
for  approval.  Some  JTFs  have  also  used  self 
developed  awards  and  certificates.  These  awards 
and  certificates  must  be  funded  from 
contributions  by  JTF  members  and  cannot  be 
funded  with  OSD  or  Service  funds. 

The  JTD  should  not  overlook  facility 
support  provided  by  the  host  installation’s 
agencies.  Other  JT&E  activities  may 
subsequently  be  located  at  the  same  facility  and 
will  require  cooperation  and  support  from  many 
of  these  same  agencies. 

E.  DISPOSITION  OF  ASSETS 

Close  down  of  the  JTF  will  involve  the 
disposition  of  two  categories  of  assets:  facilities 
and  property.  The  JTD  should  plan  to  dispose  of 
these  assets  in  accordance  with  the  appropriate 
policies  and  procedures. 
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Facilities 

The  return  of  the  facilities  will  be  in 
accordance  with  procedures  coordinated  between 
the  JTD,  the  lead  Service  OTA  POC,  and  the  host 
base  installation  office. 

Property 

The  JTF  may  end  up  with  three  sets  of 
property  that  must  be  disposed  of:  that  provided 
or  purchased  by  the  Services,  that  provided  by 
OSD,  or  that  purchased  with  OSD  funds. 

Service  property  provided  or  purchased 
by  the  Services  remains  the  property  of  that 
Service  and  will  be  returned  to  that  Service  when 
no  longer  needed.  Return  of  the  property  should 
be  coordinated  through  the  Service  OTA  POC, 
who  will  define  disposition  procedures.  An 
inventory  of  all  such  property  will  be  conducted 
prior  to  physical  transfer  of  the  property  back  to 
the  Service. 

As  the  JTF  nears  completion  of  its  program, 
the  JTF  should  contact  the  respective  Service 
OTA  POCs  so  that  provisions  can  be  made  for 
the  transfer  of  property  to  an  organization  of  that 
Service.  In  some  instances,  another  JTF  may 
have  need  for  some  of  this  property.  If  this  is  the 
case,  the  JTF  Service  deputy  should  coordinate 
with  the  Service  OTA  POC  and  the  other  JTF 
relative  to  the  possible  transfer  of  the  equipment 
from  one  JTF  to  another.  The  OSD  JT&E 
Coordinator  should  also  be  notified  to  ensure  that 
the  property  being  transferred  is  not  part  of  a 
larger  system  of  which  a  portion  may  consist  of 
OSD  purchased  equipment. 


OSD  property  purchased  by  DDT&E,  or 
purchased  by  the  JTF  with  OSD  funds,  is  and 
remains  the  property  of  DDT&E.  All  OSD 
property  assigned  to  or  purchased  by  the  JTF, 
either  through  contracts  or  by  US  Government 
agencies  will  be  returned  to  OSD  control  when 
no  longer  required  for  the  JTF  mission.  An 
inventory  of  OSD  property  will  be  conducted 
prior  to  the  transfer  of  OSD  property. 

In  no  case  should  the  JTD  take  any  action  or 
make  any  commitments  regarding  the  disposition 
of  OSD  property  to  another  JTF,  or  other 
organization  without  prior  coordination  with 
DDT&E.  While  there  may  be  a  valid  need  for 
some  of  the  OSD  property  by  organizations 
outside  the  JT&E  community,  OSD  property  will 
not  be  transferred  to  these  organizations  without 
DDT&E  approval. 

In  cases  involving  institutionalization, 
special  provisions  may  be  made  by  OSD  and  the 
Services  for  the  temporary  reassignment  of 
specific  items  of  property  to  aid  in  the 
institutionalization  of  the  JTFs  procedures, 
methods,  or  products. 

F.  JTF  PRODUCTS 

The  JTF  will  produce  products  that  have 
value  to  areas  other  than  JT&Es.  Such  products 
include  models,  simulations,  and  databases 
produced  or  used  by  the  JTF. 

Models  and  Simulations 

If  the  JTF  has  adapted  or  developed  a  model 
for  its  use,  it  represents  a  product  of  the  JTF. 
The  lead  Service  will  retain  a  copy  of  all  models 
and  simulations  employed  in  the  JT&E  program 
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for  use  in  the  performance  of  its  mission  and  for 
issuance  to  future  JFSs  and  JTFs  that  may  have  a 
need  for  them.  A  JTF  that  acquires,  modifies  or 
develops  a  model  or  simulation  should  provide 
the  lead  Service  modeling  and  simulation  agent 
with  the  model  or  simulation  to  include  the 
documentation  (user  instructions  and  technical 
details  regarding  the  models  structure)  required 
for  subsequent  use  or  modification  of  the  model 
or  simulation. 

Databases 

In  most  cases,  considerable  JTF  efforts  were 
expended  in  the  collection  of  data  and  the 
development  of  a  database.  These  represent 
products  that  should  be  archived  and  protected 
from  corruption  to  ensure  availability  to  other 
JT&E  and  Service  programs.  All  JTF  data  and 
databases  should  be  transferred  to  the  Services. 
As  with  models  and  simulations,  each  Service 
has  a  principle  recipient  of  JT«&E  databases  and 
controls  their  transfer  to  future  JFSs,  and  JTFs. 

Documents  and  Reports 

All  technical  documents  (observations, 
findings,  recommendations,  results)  and  reports 
produced  by  the  JTF  will  be  approved  by 
DDT&E  prior  to  archiving  in  supporting  technical 
libraries.  Copies  of  all  approved  technical 
documents  and  reports  should  be  forwarded  to 
the  AFOTEC  library  for  permanent  storage.  The 
address  for  the  AFOTEC  library  is  listed  in 
Chapter  2.  Copies  of  all  DDT&E  approved 
technical  documents  and  reports  that  are  pertinent 
to  the  DoD  mission,  contribute  to  the  DoD,  or 
national  scientific  or  technological  base  should 


also  be  forwarded  to  the  Defense  Technical 
Information  Center  (DTIC).  Procedures  for 
submission  of  technical  documents  and  reports  to 
DTIC  can  be  obtained  by  contacting: 

*  ja^^biMI(TO3)  274;^^  liiiiiSI 


G .  CONTRACTS  AND  SUPPORT 

AGREEMENTS 

All  contracts  and  agreements  that  were 
established  by  the  JTF  should  be  closed  and 
some  may  require  special  actions  associated  with 
the  close  out.  The  OSD  JT&E  Coordinator  will 
be  available  to  assist  in  this  area.  An  initial  action 
required  in  the  close  out  of  a  contract  is  a  review 
relative  to  completion  date,  level  of  effort,  and 
scope.  The  contract  should  be  evaluated  in  terms 
of  the  remaining  JTF  requirements,  available 
resources,  level  of  effort  remaining,  and  the 
funding  available  to  complete  the  work.  If  it  is 
determined  that  no  additional  support  is  required 
for  the  completion  of  the  JTF  or  to  support  close 
down  plans,  action  should  be  initiated  to  close 
out  the  contract(s).  The  JTD  should  also 
terminate  all  support  agreements  with  host- 
installation  and  supporting  organizations  and 
agencies. 

H.  CLOSE  OUT  OF  FISCAL 
ACCOUNTS 

Even  after  a  JTF  closes  its  doors,  the  clean 
up  of  financial  matters  will  continue.  Late  billings 
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and  final  contract  dispositions  will  create 
requirements  for  funds  after  the  JTF  is 
disestablished.  The  OSD  JT&E  Coordinator 


will  provide  assistance  to  the  JTF  financial 
manager  in  this  area,  specifically  in  the  area  of 
the  transfer  of  fiscal  responsibilities  from  the  JTF 
to  OSD. 
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ANNEX  A 

LIST  OF  ACRONYMS 


ACC 

ADP 

AFOTEC 

APA 

C2 

C3 

CATEX 

CDP 

CINC 

CMC 

CNO 

COI 

COMSEC 

COTR 

CPO 

CPX 

CRE 

DBMS 

DCSOPS 

DC/S  M&RA 

DC/S  R&P 

DDT&E 

DESA 

DIA 

DMAP 

DoD 

DoDD 

D,T,SE&E 

DTD 

EA 

EIS 

EPA 

FCC 

FDMC 

FFRDC 

FMF 

FOA 

FONSI 

FSD 

FY 

FYTP 

GOSC 

GPS 

HQMC 

IDRL 

IPR 


JT&E  Handbook 
February  1, 1996 


Air  Combat  Command  (Air  Force) 

Automated  Data  Processing 

Air  Force  Operational  Test  and  Evaluation  Center  (Air  Force) 

Analysis  Plan  for  Assessment 

Command  and  Control 

Command,  Control  and  Communications 

Categorical  Exclusion 

Close  Down  Plan 

Commander  in  Chief 

Commandant  of  Marine  Corps  (Marine  Corps) 

Chief  of  Naval  Operations  (Navy) 

Critical  Operational  Issue 

Communications  Security 

Contracting  Officer's  Technical  Representative 

Civilian  Personnel  Office 

Command  Post  Exercise 

Consolidated  Resource  Estimate 

Database  Management  System 

Deputy  Chief  of  Staff  for  Operations  (Army) 

Deputy  Chief  of  Staff  for  Manpower  and  Reserve  Affairs  (Marine  Corps) 

Deputy  Chief  of  Staff  for  Requirements  and  Programs  (Marine  Corps) 

Deputy  Director  Test  and  Evaluation 

Defense  Evaluation  Support  Activity 

Defense  Intelligence  Agency 

Data  Management  and  Analysis  Plan 

Department  of  Defense 

Department  of  Defense  Directive 

Director,  Test,  Systems  Engineering  and  Evaluation 

Deputy  Test  Director 

Environmental  Assessment 

Environmental  Impact  Statement 

Environmental  Protection  Agency 

Federal  Communications  Commission 

Fiscal  Director  Marine  Corps  (Marine  Corps) 

Federally  Funded  Research  and  Development  Center 
Fleet  Marine  Force  (Marine  Corps) 

Field  Operation  Agency  (Air  Force  and  Army) 

Finding  of  No  Significant  Impact 
Feasibility  Study  Director 
Fiscal  Year 

Five  Year  Test  Plan  (Army) 

General  Officer  Steering  Committee 
Global  Positioning  System 
Headquarters  Marine  Corps  (Marine  Corps) 

Integrated  Data  Requirements  List 
In  Progress  Review 
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ISA 

IV&V 

JCS 

JFS 

JT&E 

JTD 

JTF 

LAN 

MAJCOM 

MCSYSCOM 

MCSYSCOM/T&E 

MCO 

MCOTEA 

MIPR 

MLM 

MOA 

MOE 

MOT&E 

MOP 

NEPA 

N091 

N91202 

NSA 

O&M 

OCA 

OPSEC 

OPTEC 

OPTEVFOR 

OSD 

OTA 

OT&E 

OTP 

PC 

PCS 

PID 

POC 

POM 

PTP 

PY 

QA 

SAC 

SCG 

SCI 

SCIF 

SME 

SOFA 

SSO 

T&E 

TAB 

TAG 

TDY 

TEIN 

TF 


Inter-Service  Support  Agreement 

Independent  Verification  and  Validation 

Joint  Chief  of  Staffs 

Joint  Feasibility  Study 

Joint  Test  and  Evaluation 

Joint  Test  Director 

Joint  Test  Force 

Local  Area  Network 

Major  Command  (Air  Force  and  Army) 

Marine  Corps  Systems  Command 

Marine  Corps  Systems  Command/Test  and  Evaluation 

Marine  Corps  Order 

Marine  Corps  Test  and  Evaluation  Agency 

Military  Interdepartmental  Purchase  Request 

Mission  Level  Measure 

Memorandum  of  Agreement 

Measure  of  Effectiveness 

Multi-Service  Operational  Test  and  Evaluation 

Measure  of  Performance 

National  Environmental  Policy  Act 

Office  of  Director  of  Test  and  Evaluation  (Navy) 

Office  of  Director  of  Tests  and  Evaluation  and  Technology  Requirements, 
Test  and  Evaluation  Division  (Navy) 

National  Security  Agency 
Operations  and  Maintenance 
Original  Classification  Authority 
Operational  Security 

Operational  Test  and  Evaluation  Command  (Army) 

Operational  Test  and  Evaluation  Force  (Navy) 

Office  of  the  Secretary  of  Defense 
Operational  Test  Agency 
Operational  Test  and  Evaluation 
Outline  Test  Plan  (Army) 

Planning  Committee 
Permanent  Change  of  Station 
Program  Initiation  Document 
Point  of  Contact 

Program  Objective  Memorandum 

Program  Test  Plan 

Program  Year 

Quality  Assurance 

Senior  Advisory  Council 

Security  Classification  Guide 

Special  Compartment  Information 

Special  Compartmented  Information  Facility 

Subject  Matter  Expert 

Status  of  Forces  Agreement 

Special  Security  Office 

Test  and  Evaluation 

Technical  Advisory  Board 

Technical  Advisory  Group 

Temporary  Duty 

Test  and  Evaluation  Identification  Number  (Navy) 

Test  Force 
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TM 

Test  Manager 

TRP 

Test  Requirements  Plan  (Air  Force) 

TSARC 

Test  Schedule  and  Review  Committee  (Army) 

TSPI 

Time,  Space,  Position  Information 

TTP 

Tactics,  Techniques,  and  Procedures 

UDS 

Universal  Documentation  System 

UIC 

Unit  Identification  Code 

WBS 

Work  Breakdown  Structure 
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ANNEX  B 

GLOSSARY  OF  TERMS 


Analysis  Plan  for  Assessment  (APA) 

A  document  published  by  the  JFS  team  that 
stipulates  what  actions  must  be  accomplished  by 
a  JTF  in  order  to  collect  and  analyze  sufficient 
data  to  resolve  the  JT&E  issues  and/or  concepts. 
The  APA  specifies  what  must  be  done  in 
sufficient  detail  so  that  the  JFS  team  can  quantify 
the  resources  required  to  accomplish  the  JT&E. 

Battalion  Training  Strategy  (OPTEMPO) 
The  actual  cost  of  parts  and  fuel  to  operate 
annual  miles  for  vehicles  and  hours  for  aircraft 
executing  training  in  the  field  army. 

Critical  Operational  Issues  (COI) 

The  major  issues  (questions/problems)  that 
a  JT&E  is  addressing,  usually  phrased  in  the 
form  of  questions. 

Data  Management  and  Analysis  Plan 
(DMAP) 

A  program  level  document  that  details  what 
data  will  be  collected,  how  it  will  be  cataloged, 
stored,  controlled,  analyzed,  and  archived.  The 
program  DMAP  also  contains  detailed  procedures 
for  these  functions.  A  DMAP  may  also  be 
required  to  detail  what  and  how  data  will  be 
collected,  managed,  and  analyzed  for  a  specific 
test  activity. 

Defense  Evaluation  Support  Activity 
(DESA) 

A  DoD  support  organization  available  to 
JFS  and  JTF  teams  for  advice  and  support 
regarding  virtually  all  aspects  of  JT&E. 


Designated  Support  Agent 

The  organization  or  agency  that  is 
designated  by  competent  authority  to  provide 
support  to  a  JT&E.  Frequently  referred  to  as  the 
support  agent. 

Environmental  Assessment  (EA) 

A  study  required  by  the  National 
Environmental  Policy  Act  (NEPA)  to  determine  if 
significant  environmental  impacts  are  expected 
from  a  proposed  activity. 

Environmental  Impact  Statement  (EIS) 

A  report  required  by  NEPA  that  describes 
the  environmental  consequences  of  a  proposed 
activity. 

Feasibility  Study  Team 

The  personnel  who  perform  a  JFS. 

Feasibility  Study  Director  (FSD) 

The  person  appointed  as  responsible  for  the 
conduct  of  the  JFS. 

Free  Play 

Activities  of  a  player  staff  as  it  responds  to 
stimuli  provided  by  the  control  structure  of  a 
"game,"  where  the  stimuli  are  logical 
consequences  of  previous  player  actions.  In  this 
context,  a  game  is  some  representation  of  the  real 
world  intended  as  context  for  training  of  the 
player  staff. 

General  Officer  Steering  Committee 
(GOSC) 

A  group  of  General  Officers  from  the 
Services  interested  in  a  particular  JT&E  issue  or 
operational  concept  who  are  invited  by  the  JTD  to 
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advise  on  issues  of  doctrine,  policy,  or  tactics. 
The  intent  is  to  capture  and  integrate  Service 
representation  in  the  JT&E  at  a  senior  officer 
level. 

Instrumentation 

Equipment  used  during  a  test  activity  to 
capture  and  record  data. 

Issue 

A  question  that  a  nomination  poses  that  a 
JT&E  will  resolve.  The  measures  and  data 
elements  are  designed  to  resolve  the  issues. 

Joint  Feasibility  Study  (JFS) 

A  formal  study  undertaken  to  determine 
whether  a  proposed  JT&E  should  be  chartered 
for  execution  by  a  Joint  Test  Force  (JTF). 

Joint  Test  Director  (JTD) 

The  person  appointed  as  responsible  for 
executing  a  charted  JT&E  and  directing  the 
efforts  of  the  JTF.  Should  be  at  least  an  0-6 
(possibly  GS-15)  with  test  and  evaluation 
experience  and  a  background  in  the  technical 
aspects  of  the  JT&E. 

Joint  Test  and  Evaluation  (JT&E) 

Those  activities  dedicated  to  addressing  an 
issue  or  concept  that  was  nominated  by  OSD,  the 
Joint  Staff,  the  CINCs,  or  the  Services  and  has 
been  chartered  by  OSD  to  be  conducted  by  a 
JTF. 

Joint  Test  and  Evaluation  (JT&E) 
Program 

Those  activities  managed  by  DDT&E  for 
the  DoD,  including  all  active  and  proposed  JFSs 
and  JTFs.  Frequently  confused  with  the  actions 
of  a  JTF,  which  are  to  conduct  a  JT&E  dealing 
with  an  issue  or  concept,  the  JT&E  program 
includes  all  functions  that  support  the  JT&E  such 
as  the  convening  and  support  of  the  JT&E 


Planning  Committee,  SAC,  TAB,  GOSC,  and 
program  and  budget  functions. 

Joint  Test  Force 

A  formal  organization  lead  by  a  JTD  and 
staffed  by  the  Services  for  a  specific  time  period 
to  conduct  a  JT&E  under  the  auspices  of  an  OSD 
charter. 

Measure  of  Effectiveness  (MOE) 

A  quantifiable  entity  that  expresses  the 
effectiveness  of  a  system  or  concept  under  test. 
An  MOE  can  also  be  defined  as  an  algorithm  that 
uses  data  to  compute  a  quantity  called  the 
measure. 

Mission  Level  Measure  (MLM) 

A  quantitative  or  qualitative  measure  of  a 
system's  capabilities  or  characteristics  in  terms  of 
their  effect  on  the  mission  of  which  the  system  is 
a  part. 

Measure  of  Performance  (MOP) 

A  quantitative  or  qualitative  measure  of  a 
system's  capabilities  or  characteristics. 

Nomination 

The  process  used  to  bring  joint  issues  and 
concepts  to  the  attention  of  the  DDT&E. 

Objectives 

Organizational  planning  tools  that  focus 
attention  on  those  areas  will  be  tested  to  resolve 
the  issues  or  concepts.  JT&E  progress  can  be 
measured  in  terms  of  attaining  the  test  objectives. 

Outline  Test  Plan  (OTP) /Test  Resource 
Plan  (TRP) 

Those  resource  requirements  documents 
used  by  the  Army  and  Air  Force  respectively  for 
users  to  specify  personnel  and  equipment 
requirements  to  be  used  in  the  support  of  joint 
test  activities. 
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Piggyback 

A  form  of  testing  where  a  joint  test  force 
uses  deployed  resources  of  one  or  more  Services 
for  testing  before,  during,  or  after  a  concurrently 
scheduled  Service  or  Joint  Staff  training  exercise 
in  order  to  collect  data. 

Program  Analysis  Methodology 

That  portion  of  the  APA  that  defines  what 
analysis  must  be  done  using  the  collected  data 
elements  to  address  all  of  the  issues. 

Program  Test  Plan  (PTP) 

A  document  that  specifies  in  detail  the 
procedures  for  conducting  test  activities; 
collecting,  processing,  verifying,  storing,  and 
transporting  the  required  data;  and  producing  the 
required  reports. 

Quick-Look 

Those  procedures  established  to  assure  the 
JTD  that  the  amount  and  quality  of  data  being 
collected  during  test  activities  is  adequate. 

Reconstruction 

A  posttest  analysis  process  used  to  verify 
the  accuracy  of  collected  data  by  alignment  of  the 
data  to  test  activities. 

Senior  Advisory  Council  (SAC) 

An  advisory  body  that  reviews  selected 
nominations,  the  results  of  JFSs  and  JTFs,  and 
recommends  appropriate  actions  to  the  DDT&E. 

Service  Deputy 

A  person  appointed  by  a  participating 
Service  to  participate  in  a  JTF.  This  person 


serves  as  a  functional  member  of  the  JTF  while 
representing  the  interest  of  the  appointing 
Service.  The  Service  Deputy  should  be  an  05  or 
06  with  test  and  evaluation  experience  and  a 
background  in  the  technical  aspects  of  the  JT«&E. 

Technical  Advisor 

A  JTF  member  designated  by  the  JTD  to 
advise  on  technical  matters  and  to  resolve 
technical  differences  of  opinion  within  the  JTF. 
The  technical  advisor  is  responsible  for  keeping 
JT&E  activities  focused  on  the  chartered  concepts 
and/or  issues. 

Technical  Advisory  Board  (TAB) 

A  group  of  senior  scientists,  engineers,  and 
analysts  who  advise  the  DDT&E,  SAC,  JT&E 
PC,  FSDs,  and  JTDs  on  technical  matters 
relevant  to  JT&Es. 

Technical  Advisory  Group  (TAG) 

An  advisory  body  formed  to  provide  direct 
technical  support  and  advice  to  a  JTF.  The  TAG 
composition  is  similar  to  the  TAB  but  is  not  as 
senior. 

Test  Activity 

A  collective  term  used  to  describe  a  series 
of  related  tests  or  studies  conducted  to  collect  and 
analyze  data.  Test  activities  can  range  from 
analysis  using  studies  and  models  to  field  tests 
that  involve  deployed  combat  units. 

Test  Manager 

A  JTF  member  assigned  by  the  JTD  to  be 
responsible  for  the  planning,  execution,  and 
reporting  of  a  specific  test  activity. 
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POCs  for  JT&E  SUPPORT 


OSD 


JCS 


USAF 


USA 


USN 


USMC 


DESA 


DDT&E 

Tel  -  (703)  697-3406 

FAX  -  (703)  614-7040 

E-Mail  LEDESMAR@ACO.OSD.MIL 


Joint  Staff/J8 

Tel  -  (703)  694-9759 

FAX  -  (703)  694-6601 


USAF/TEP 

Tel  -  (703)  695-0900 

FAX  -  (703)  695-0803 

E-Mail  BRECHWAJ@TEP.HQ.AF.MIL 


DCSOPS  (DAMO-FDR) 

Tel  -  (703)  697-4044 
FAX  -  (703)  614-2675 

E-Mail  AGOSTA@PENTEMH8.ARMY.MIL 


OP  912D2 

Tel  -  (703)  697-1485/0181 
DSN  -  227-1485/0181 
FAX  -  (703)  697-1070 


MCSYSCOM  Code(TE) 

Tel  -  (703)  784-5964 
FAX  -  (703)  784-3432 

E-Mail  MEADP@MQG-SMTP3.USMC.MIL 


DESA-BMC 

Tel  -  (505)  262-4529 

Fax-  (505)  260-2759 

E-Mail  HOLMESB@DESA.OSD.MIL 


JT&E  Coordinator 

Tel  -  (703)  578-6581 

Fax  -  (703)  578-6583 

E-Mail  BLOOMELE@ACO.OSD.MIL 


AFOTEC/XRJ 

DSN  -  246-5339 

Tel  -  (505)  846-5339 

FAX  -  (505)  846-5214 

E-Mail  MAGEEW@P3.AFOTEC.AF.MIL 

OPTECCSTE-OPM 

Tel  - (703)  681-6518 

DSN  -  289-6518 

FAX  -  (703)  681-7584 

E-Mail  YOUNGA@OPTECI.ARMY.MIL 
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ANNEX  D 

RELATED  JT&E  DOCUMENTS 


DoD  Instruction  5000.2 

This  DoD  Instruction  provides  the  basic 
framework  for  the  Test  and  Evaluation  of 
programs  sponsored  or  supported  by  OSD. 

DoD  Manual  5000.3*M-4 

This  manual  provides  a  description  of  the 
joint  test  and  evaluation  nomination  and  selection 
process,  describes  the  organizational  framework 
within  each  Service  that  supports  the  program 
and  identifies  principal  participants  and  their 
respective  responsibilities. 

DoD  Manual  7110.1-Ml 

This  manual  outlines  the  procedures  for  the 
distribution  and  use  of  OSD  funds  in  support  of 
OSD  sponsored  or  supported  projects. 

DoD  Manual  5220.22-M 

This  manual  provides  detailed  information 
relative  to  the  Government  Information  Security 
Program  for  safeguarding  classified  information. 

DoD  Directive  3200. 11-D 

This  directive  is  a  summary  of  Major  Range 
and  Test  Facility  capabilities. 

DoD  Directive  3200.12 

This  directive  outlines  the  mission, 
responsibilities,  and  functions  of  the  Defense 
Technical  Information  Center  (DTIC).  Copies  of 
all  JFS  generated  technical  documents  and 
readers  should  be  forwarded  to  DTIC. 

DoD  Directive  4120.14 

This  directive  establishes  guidelines  relative 
to  Environmental  Concerns  for  DoD  programs. 


DoD  Directive  5230.24 

This  directive  establishes  the  requirement 
for  all  managers  of  technical  programs  to  assign 
distribution  statements  to  technical  documents 
generated  within  their  program. 

DoD  Directive  6050.1 

This  directive  is  a  list  of  activities  that  have 
previously  been  found  to  have  no  detrimental 
effects  on  the  environment  and  do  not  require  an 
EAorEIS. 

DIA  Regulation  55-3 

This  regulation  is  a  guide  for  obtaining 
threat  information  support. 

DoD  Regulation  5200. 1-R 

This  regulation  establishes  requirements 
relative  to  the  government  information  security 
program. 

Memorandum  of  Agreement  on  Multi- 
Service  Test  and  Evaluation  and  Joint 
Test  and  Evaluation 

This  MOA  provides  the  basic  framework 
for  T&E  conducted  by  two  or  more  Operational 
Test  Agencies. 

Joint  Feasibility  Study  Handbook 

This  handbook  consolidates  OSD  guidance 
and  direction,  information,  references,  and 
procedures  on  conducting  a  Joint  Feasibility 
Study. 

Joint  Test  and  Evaluation  Handbook 

This  handbook  consolidates  OSD  guidance 
and  direction,  information,  references,  and 
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procedures  on  conducting  a  Joint  Test  and 
Evaluation. 

AFOTEC  Instruction  99-101 

This  Instruction  provides  the  guidelines  and 
procedures  for  the  AFOTEC  conduct  of  OT&E 
on  Air  Force  systems.  The  instruction  contains 
guidance  relative  to  resource  planning  and 
funding  of  Joint  Test  and  Evaluations. 

AFOTEC  Pamphlet  55-8 

This  pamphlet  is  a  guide  for  resource 
managers  and  test  managers  on  procedures  for 
preparing  the  Test  Resource  Plan  (TRP). 

Army  Regulation  15-38 

This  regulation  provides  guidance  and 
establishes  procedures  governing  the  Test 
Schedule  and  Review  Committee  (TSARC). 


Test  Schedule  and  Review  Committee 
Handbook 

This  handbook  provides  TSARC  members 
and  the  user  test  community  with  a  concise 
document  that  combines  information  from  many 
sources  for  use  in  preparing  Outline  Test  Plans 
(OTPs)  for  inclusion  in  the  Department  of  Army 
Five  Year  Test  Program  (FYTP) 

Marine  Corps  Order  5000.11B 

This  Order  establishes  guidance  for  the  Test 
and  Evaluation  of  systems  and  equipment  to  be 
employed  by  the  Marine  Corps. 

Military  Standard  1806 

This  standard  provides  procedures  for 
marking  scientific,  technical,  engineering, 
production,  and  logistics  technical  data,  to  denote 
the  extent  to  which  they  are  available  for 
secondary  distribution. 
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ANNEX  E 
TEST  METHODS 


A.  INTRODUCTION 

This  annex  describes  the  various  test  methods 
that  a  JT&E  could  employ,  in  effect  the  tools  that 
are  available  to  address  the  program  issues.  The 
list  of  methods  is  lengthy,  but  not  exhaustive. 
There  are  two  basic  kinds  of  tools  for 
accomplishing  the  JT&E  efforts.  The  two  basic 
methods  are  1)  problem  solving,  and  2)  testing 
(e.g.  assessments,  evaluations,  or  validations). 
Some  of  the  relative  costs  and  benefits  associated 
with  each  method  are  discussed  in  the  following 
paragraphs  and  are  summarized  in  Figure  E-1. 
The  advisability  of  using  one  or  more  of  the 
methods  will  depend  on  specific  test 
requirements  and  goals  and  on  such  factors  as 
availability,  credibility,  cost,  acceptance  of  test 
results  by  the  Services,  fidelity  of  the  test  data, 
test  site  location,  and  available  support. 

The  JFS  will  make  an  initial  determination 
of  test  methods  as  part  of  the  PTD  development. 
JTF  coordination  for  the  actual  use  of  the 
resources  to  conduct  the  test  activities  may 
determine  that  some  are  not  available  when 
needed  or  some  are  too  costly.  If  this  occurs,  the 
JTF  must  quickly  decide  on  alternatives  that 
could  include  the  identification  and  use  of  other 
test  methods. 

B .  PROBLEM  SOLVING  EFFORTS 

Most  JT&Es  have  at  least  one  issue  that 
calls  for  fixing  a  problem  or  finding  a  better  way. 
Thus,  the  classic  problem  solving  process  is 
involved  in  almost  all  JT&Es.  This  includes 
bringing  together  people  capable  of  solving  the 
problem,  brainstorming  for  solutions,  analyzing 


and  ranking  solutions,  and  selecting  the  best 
solution.  The  magnitude  of  this  effort  can  vary 
widely  with  the  complexity  of  the  problem. 

A  primary  step  in  most  problem  solving 
activities  is  information  collection.  Reviews  of 
published  documentation  is  the  most 
straightforward  way  of  collecting  data  and  there 
are  many  excellent  sources  available  to  the  JTF. 
These  include  previous  test  and  exercise  results, 
industrial  papers  and  studies,  FFRDC  studies 
and  analyses,  and  Service  programs  in  the  same 
technical  field.  Operational  records  are  another 
good  source  of  information.  A  drawback  to 
using  operational  data  is  that  it  may  require 
manipulation  of  the  data  to  reveal  the  information 
of  interest  and  may  lack  the  fidelity  or  accuracy 
needed. 

New  or  supplemental  information  may  be 
acquired  through  the  use  of  surveys  or  expert 
opinion,  or  the  use  of  validated  models  and 
simulations.  This  can  increase  the  effectiveness 
of  operation  records  by  filling  in  those  areas 
where  operational  records  are  weak.  Validated 
modeling  and  simulations  can  also  be  used  to 
develop  new  information  by  looking  at  existing 
data  in  new  or  different  ways. 

When  research,  modeling,  or  simulations 
do  not  provide  the  data  needed,  laboratory 
measurements  may  be  required.  With  laboratory 
measurements,  the  resulting  data  are  not 
compared  (evaluated)  against  a  set  of  criteria. 
Rather,  the  data  are  simply  collected  and 
reported. 
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SELECTION 

FACTORS 

TEST 

VEHICLES  TOOLS 

0^ 

/ 

i 

LABORATORY  TEST 

1 

1 

1 

1 

MODELING  &  SIMULATION 

2-3 

2-3 

2 

MINI-TESTS 

1 

2 

2 

1 

CPX 

3 

3 

3 

4 

FIELD  TEST  (Piggyback) 

2 

4 

2-3 

3 

FIELD  TEST  (Dedicated) 

2 

3 

4 

1 

NOTE:  Values  are  subjective  estimates  where  1  represents  the  most 
desirable  condition 


Figure  E-1.  Nominal  Value  of  Testing  Options 


C.  ASSESSMENT,  EVALUATION,  OR 
VALIDATION 

JT&Es  are  typically  associated  with 
activities  to  collect  and  analyze  data  to  assess, 
evaluate,  or  validate  a  concept  or  resolve  an 
operational  issue.  Frequently,  JT&E  team 
members  who  acquired  their  testing  experience 
with  a  Service  test  organization  tend  to  think  of 
testing  only  in  regards  to  new  Service  systems, 
i.e.  hardware  and  software.  In  reality,  this  focus 
is  too  narrow.  For  example,  the  interoperability 
of  two  or  more  systems  can  be  assessed;  the 
effectiveness  of  an  operational  procedure  can  be 
evaluated;  or,  a  new  or  revised  process  can  be 
validated.  JTF  staff  members  must  think  in  the 
broadest  terms  of  testing  and  how  test  activities 
will  contribute  to  the  resolution  of  the  JT&E 
issues. 


An  assessment  provides  an  understanding 
of  the  relative  magnitude  of  what  is  involved  and 
determines  the  size  and  importance  of  that  which 
is  being  examined  in  terms  of  a  whole  picture. 
An  evaluation  determines  the  relative  worth  or 
significance  of  that  which  is  being  evaluated 
(there  are  usually  criteria  which  are  used  to  aid  in 
the  evaluation  process).  A  validation  provides  a 
means  for  corroborating  or  supporting  something 
with  a  sound  basis.  Validation  creates  the 
foundation  that  ties  the  concept  being  validated  to 
an  authority  on  the  subject. 

The  following  sections  discuss  the  various 
methods  for  obtaining  data  to  support 
assessments,  evaluations,  or  validations. 
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Surveys 

The  term  surveys  is  used  to  cover  a  range 
of  methods  including  questionnaire  and  interview 
methods  to  collect  data  about  system 
performance.  Frequently,  survey  data  involves 
the  use  of  experts  but  may  also  be  used  to  elicit 
information  from  typical  users  of  a  system. 
Surveys  are  useful  when  evaluation  criteria  that 
are  ill-defined,  such  as  when  gauging  general 
improvements  of  a  process  or  assessing  the 
extent  of  a  problem.  Surveys  are  also  useful 
when  the  feature  under  scrutiny  does  not  lend 
itself  to  objective  analysis,  such  as  human  factor 
concerns. 

The  cost  for  conducting  surveys  are 
typically  much  lower  than  assessment  or 
evaluation  methods.  One  factor  that  must  be 
considered,  however,  is  the  cost  of  access  to  the 
survey  participants.  The  fidelity  of  a  survey  is 
totally  dependent  on  the  quality  and  quantity  of 
the  responses  from  the  survey  participants. 

Laboratory  Tests 

Laboratory  tests  are  generally  the  least 
expensive  and  least  complex  type  of  tests  to  plan 
and  execute.  They  are  generally  used  to  obtain 
engineering  characteristics  or  specifications 
relative  to  the  performance  of  specific  systems 
under  laboratory-controlled  conditions.  Such 
tests  can  range  from  the  use  of  bench  tests,  to 
anechoic  chambers,  to  antenna  measurement 
facilities,  and  to  system  stress  test  facilities. 

In  addition  to  being  perhaps  the  least 
expensive  and  simplest  test  to  plan  and  conduct, 
lab  tests  provide  the  means  to  exercise  the 
greatest  control  over  the  test  and  can  be 
conducted  at  whatever  pace  is  necessary  to 
confirm  and  validate  the  data  collected. 
Equipment  can  be  left  in  the  testconfiguration 


until  the  data  are  validated,  and  if  the  test  needs  to 
be  rerun,  the  equipment  is  already  setup  and 
ready  for  repeat  testing.  Nearly  all  of  the  testing 
conditions  can  be  rigidly  controlled  to  preclude 
outside  interference  on  system  operations  and  the 
data  collection  and  management  process.  Testing 
can  be  conducted  based  on  the  availability  and 
operating  hours  of  the  test  facility.  Resources  for 
conducting  the  test,  to  include  test  equipment  and 
support  personnel,  are  generally  available  as  part 
of  the  test  facility  support  agreement.  Costs  for 
this  type  of  testing  will  be  those  associated  with 
the  transportation  of  equipment  and  systems  to 
and  from  the  test  location  and  the  TDY  costs  of  a 
test  team  from  the  JTF. 

In  most  instances,  lab  testing  alone  is 
insufficient  to  resolve  program  issues,  since  the 
results  generally  do  not  reflect  the  system’s 
performance  in  an  operational  environment  (i.e., 
the  system’s  performance  within  operational 
profiles  and  employed  by  operators  of  varying 
degrees  of  experience  attempting  to  accomplish 
an  operational  task).  However,  lab  testing  can  be 
a  valuable  tool  for  obtaining  operating  insights 
and  data  on  a  system  prior  to  moving  into  a  field 
test  where  the  collection  of  data  are  more 
difficult.  In  this  regard,  the  data  obtained  from 
lab  tests  will  ordinarily  be  used  later  in  the  JT&E 
in  conjunction  with  data  from  other  types  of 
testing  to  address  the  program  issues  and  in 
designing  other  tests,  to  include  mini-tests  and 
field  tests. 

Modeling  and  Simulation 

Models  and  simulations  can  vary 
significantly  in  size  and  complexity  and  can  be 
useful  tools  in  several  respects.  They  can  be 
used  to  conduct  predictive  analyses  for 
developing  plans  for  test  activities,  for  assisting 
test  planners  in  anticipating  problem  areas,  and 
for  comparison  of  predictions  to  collected  data. 
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Validated  models  and  simulations  can  also  be 
used  to  examine  test  article  and  instrumentation 
configurations,  scenario  differences,  conduct 
Tv/zat-r/ tradeoffs  and  sensitivity  analyses,  and  to 
extend  test  results. 

One  sequence  that  has  proved  useful  begins 
with  model  use  to  predict  test  results.  Tests  are 
then  used  to  validate  these  the  model  predictions, 
and  finally,  the  validated  model  is  used  to  extend 
the  test  results  to  cases  not  tested. 

The  JTF  must  pay  particular  attention  to  the 
maturity,  community  acceptance,  and 
characteristics  of  models  being  considered. 
Model  development  is  expensive.  Similarly,  a 
model  that  requires  extensive  modification  or 
completion  (most  models  can  be  improved)  may 
be  too  expensive  or  time  consuming  for 
consideration. 

A  resource  for  the  JTF  to  begin 
examination  of  potential  models  applicable  to  the 
JT&E  is  the  Joint  Staff  Catalog  of  Models, 
Simulations  and  Games. 

Mini-Tests 

Mini-tests  are  small  scale  tests  involving 
one-on-one  or  one-on-few  systems  conducted 
under  operational  or  near-operational  conditions. 
They  are  used  to  collect  engineering  data,  system 
data,  procedural  data,  or  data  on  systems  where 
personnel  must  be  in  the  decision-making  loop. 
Mini-tests  are  generally  dedicated  tests  that  afford 
complete  control  by  the  JTF.  The  costs  of  mini¬ 
tests  can  vary  significantly.  Generally,  they  are 
more  expensive  than  lab  tests  but  are  cheaper 
than  field  tests.  In  some  cases,  combinations  of 
lab  tests  and  mini-tests  can  provide  sufficient  data 
to  satisfy  program  level  issues. 


Mini-tests  have  been  used  by  JT&Es  to 
evaluate  proposed  instrumentation  schemes,  data 
collection  and  processing  techniques,  tools,  and 
procedures  for  larger  scale  field  tests.  The  use  of 
mini-tests  allows  the  team  to  evaluate  and  revise 
instrumentation,  equipment  and  procedures,  and 
train  personnel  to  perform  the  required  data 
collection  tasks,  thus  reducing  the  risk  of  failure 
and  the  associated  expense  (time  and  dollars) 
during  more  expensive  field  testing. 

As  with  costs,  the  planning  efforts  for  a 
mini-test  can  vary.  In  most  cases,  the  planning 
efforts  are  extensive  since  most  mini-tests  are 
conducted  at  Service  test  facilities  and  require 
integrating  test  facility  resources,  JT&E 
resources,  and  Service  resources. 

Command  Post  Exercises  (CPXs) 

CPXs  are  tests  or  exercises  that  are 
primarily  procedure  oriented  and  do  not  involve 
extensive  deployment  of  forces.  For  example, 
resolving  an  issue  of  the  targeting  of  air  defense 
elements  between  ground  elements  and  tactical  air 
might  be  addressed  during  a  CPX.  Such  a  CPX 
would  involve  measurements  of  the  time  it  takes 
for  an  assembled  staff  to  receive  guidance  or 
orders,  process  targeting  information,  plan 
reactions,  coordinate  the  planning  and  dispatch 
task  orders.  The  JTF  staff  should  evaluate 
potential  CPXs  against  the  data  required  to 
resolve  test  issues.  CPX  planning  and  execution 
should  keep  in  mind  that  coordination  activities 
usually  occur  within  the  play  of  a  CPX,  rather 
than  before  or  more  preferably  after  it. 
Therefore,  the  use  of  CPXs  requires  carefully 
planned  data  collection  (by  use  of  transparent 
instrumentation)  procedures  to  assure  that  the 
required  data  are  captured  in  a  free  play 
environment  and  that  data  collection  does  not 
interfere  with  the  CPX. 
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Field  Tests  and  Exercises 

Field  tests  and  exercises  are  generally  large- 
scale  activities  conducted  to  obtain  or  validate 
data  involving  concept  or  system  performance 
under  near-operational  conditions.  They  will 
range  from  one-on-one  to  many-on-many 
situations  where  data  are  required  from  many 
different  systems  interacting  simultaneously 
under  operational  conditions.  These  situations 
may  include  man-in-the-loop  decisions  that 
impact  system  performance  or  the  outcome  of 
system  interactions.  Field  tests  and  exercises  are 
usually  the  most  complex  and  expensive  that  a 
JT&E  will  conduct  and,  because  of  their  size  and 
expense,  will  require  extensive  planning  and 
coordination. 

The  JTF  should  examine  the  Joint  Staff 
Five  Year  Exercise  Schedule  and  Service  field 
test  and  exercise  schedules  to  identify 
opportunities  to  use  already  scheduled  field  tests 
and  exercises  as  test  vehicles.  Criteria  for 
identifying  field  tests  that  could  be  used  as  test 
vehicles  should  include  test  schedule,  scenario, 
location,  forces  (red  and  blue),  systems,  primary 
objectives,  magnitude,  and  cost.  Early 
coordination  with  Joint  Staff  and  Service  field 
test  and  exercise  planners  is  essential  to  the 
development  of  a  credible  JT&E  schedule. 

There  are  two  basic  types  of  JT&E  field 
tests,  piggybacked  and  dedicated.  Piggybacking 
refers  to  the  method  of  collecting  relevant  JT&E 
data  during  already  established  Service  or  joint 
tests  or  exercises.  This  is  distinct  from  a 
dedicated  test,  which  would  be  a  test  designed 
and  controlled  by  the  JT&E  for  the  sole  purpose 
of  answering  the  program  issues.  The  costs  and 
benefits  of  these  two  approaches  must  be 
weighed  in  relation  to  specific  test  goals  and 
resources.  Complete  piggybacking  may  provide 


the  lowest  cost  and  data  gathered  under  the  most 
realistic  operational  conditions.  However,  the 
inability  to  control  procedures  and  variables  and 
the  inability  to  iterate  or  repeat  test  events  may 
obviate  these  apparent  advantages. 

In  actuality,  there  is  a  more  desireable 
compromise  between  these  extremes.  That  is, 
conducting  JT&E  testing  in  conjunction  with  a 
field  test  or  exercise  during  a  dedicated 
"window. "  During  this  window,  test  resources 
are  turned  over  to  the  control  of  the  JT&E  and 
test  scenarios  and  variables  may  be  modified  to 
meet  test  objectives.  This  method  has  proven  to 
be  an  excellent  compromise  between  the  higher 
costs  of  dedicated  tests  and  the  need  to  exercise 
adequate  control.  The  JT&E  may  be  able  to  offer 
the  test  or  exercise  sponsors  an  incentive  for  this 
arrangement  by  providing  assets  (manpower, 
equipment,  and  test  results)  that  will  benefit  the 
exercise  or  lower  overall  costs. 

If  the  JT&E  only  needs  to  participate  during 
part  of  the  test  or  exercise,  or  is  going  to 
implement  testing  during  a  dedicated  window, 
then  a  decision  will  have  to  be  made  between  the 
JTF  and  the  sponsoring  agency  as  to  the  best 
time  to  conduct  the  JT&E  test  activities. 
Piggybacking  on  the  front  of  the  exercise  has  the 
advantage  of  having  fresher  troops  and  the 
greater  possibility  of  obtaining  additional  time  if 
needed.  However,  training  exercises  may  start 
with  a  great  deal  of  confusion  and  equipment 
problems  as  well  as  less  trained  participants. 
Starting  too  late  in  the  exercise  runs  the  risk  of 
tired  or  poorly  motivated  participants  who  are 
less  than  willing  to  stay  late  to  obtain  test  data. 
Choosing  a  time  near  the  middle  of  the  test  or 
exercise  may  be  the  best  bet,  sometime  after 
things  are  expected  to  be  running  smoothly  with 
enough  time  remaining  for  additional  testing  if 
required. 
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ANNEX  F 
DESA  SUPPORT 


A.  INTRODUCTION 

DDT&E  is  responsible  for  the  overall 
management  and  oversight  of  the  JT&E  program. 
This  handbook  makes  reference  to  capabilities 
that  are  available  to  support  a  JT&E.  This  annex 
provides  an  overview  of  the  specific  DESA 
capabilities  that  may  be  available.  Support  will 
be  obtained  through  the  OSD  JT&E  Coordinator. 

B.  SPECIFIC  DESA 

RESPONSIBILITIES 

•  Coordinates  disposition  of  JT&E 
resources/assets.  Provides  life-cycle 
management  of  OSD  resources  procured 
by /or  used  for  JT&E  programs. 

•  As  requested  by  the  JT&E,  provides 
technical  and  administrative  management 
services  to  JT&E  activities,  including 
financial  management  assistance, 
communications  and  computer 
management,  contract  management,  and 
security  and  environmental  consulting. 

•  As  requested  by  DDT&E  provides  an 
Independent  Evaluation  of  the  technical 
and  operational  aspects  of  the  JT&E  test 
plan  execution,  data  analysis,  and 
reporting. 

C.  FUNCTIONAL  SUPPORT 

CAPABILITIES 

Although  not  inclusive,  the  following 
support  capabilities  can  be  made  available.  The 
JTD  should  contact  the  OSD  JT&E  Coordinator 
for  more  information  relative  to  obtaining  this 


support.  If  support  is  required,  the  JTD  should 
initiate  a  Program  Identification  Document  (PID) 
with  DESA  to  formally  establish  the  requirement. 

Logistics  Support 

DESA  is  the  custodian  of  DDT&E 
investments  in  facilities,  threat  assets,  simulators, 
instrumentation,  test  equipment,  data  processing 
equipment,  and  other  peripherals  that  have  been 
purchased  by  OSD  with  JT&E  funds.  Some  of 
these  assets  may  be  available  for  JT&E  support. 
Logistics  support  capabilities  may  also  include 
maintenance,  storage,  configuration  management 
and  control,  warehousing,  spares  support, 
inventory  control,  transportation,  calibration, 
O&M,  equipment  training,  and  expertise  in 
logistics  support. 

Environmental  Support 

Support  in  this  area  can  cover  the  spectrum 
from  examination  of  the  operational  impacts  of 
the  testing  to  assistance  in  planning  to  ensure 
compliance  with  NEPA  standards  and 
requirements.  This  assistance  can  be  particularly 
beneficial  when  conducting  cost/benefit  tradeoffs 
and  identifying  long  lead  items  associated  with 
Environmental  Assessments  (EAs)  and 
Environmental  Impact  Statements  (EISs).  DESA 
maintains  a  listing  of  environmental  directives 
and  regulations  and  can  provide  assistance  in 
recommending  which  should  be  reviewed  for 
applicability.  Additionally,  DESA  can  assist  in 
the  identification  of  instrumentation  that  may  be 
required  for  the  environmental  control  of  test 
activities. 
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Security  Support 

Personnel  are  available  with  experience  in 
all  aspects  of  security  to  include  SSO,  SCI, 
SSGs,  OPSEC,  and  COMSEC.  This  support 
could  include  assistance  in  the  development  of 
procedures  to  guarantee  the  security  of  systems, 
equipment,  plans,  operations,  data,  and  reports; 
obtaining  special  security  billets  and  clearances 
for  personnel;  the  use  of  SCIF  capabilities;  the 
development  of  Security  Classification  Guides; 
the  establishment  of  interfaces  with 
compartmented  projects;  and  periodic  reviews  of 
security  procedures. 

Contracting  and  Procurement  Support 

The  DESA  Director  is  head  of  a  DoD 
warranted  contracting  activity  and  the  expertise 
associated  with  this  service  is  available  to  the 
JTF.  This  includes  the  development  and 
administration  of  contracts  to  support  JT&E 
activities,  the  identification  of  unique  resource 
requirements,  and  the  correlation  of  support 
requirements  with  responsible  agencies. 

DESA  has  existing  contracts  with  defense 
contractors  who  have  extensive  JT&E 
experience.  These  contracts  are  available  for  use 
by  a  JTF  and  a  subtask  can  be  developed  quickly 
to  obtain  the  required  support.  If  contracting 
support  is  required,  the  JTD  should  appoint  a 
Contracting  Officers  Technical  Representative 
(COTR)  from  the  JTF  staff  who  will  be 
responsible  for  providing  technical  guidance  to 
the  contractor  and  technical  inputs  relative  to 
contractor  performance  to  the  Contracting  Officer 
representative.  In  addition  to  providing 
information  and  access  to  contractors  with  JT&E 
experience,  DESA  can  provide  COTR  training 
for  JTF  personnel. 


Technical  Support 

DESA  can  provide  ad  hoc  technical  support 
that  includes  commenting  on  or  contributing  to 
the  development  of  test  plans,  data  management 
and  analysis  plans,  and  any  of  the  required 
annexes.  DESA  personnel  have  backgrounds  in 
T&E  and  work  with  and  support  on-going  JFS 
and  JTF  efforts.  Defense  contractors  under 
contract  to  DESA  have  personnel  with  JT&E 
experience,  to  include  past  JTF  Directors, 
Service  Deputies,  Chiefs  of  Staff,  Directors  of 
Operations,  and  Directors  of  Operations 
Analysis.  Technical  support  is  available  in  the 
following  areas: 

•  Program  level  guidance,  assistance,  and 
advice  in  the  development  of  JT&E  plans 
and  documentation. 

•  Reviews  of  concepts  and  plans  developed 
by  the  JTF. 

•  Analysis,  engineering,  and  T&E 
expertise. 

•  Information  and  advice  on  databases  and 
models. 

•  Information  and  advice  on  range 
equipment,  test  ranges,  and  facilities. 

•  Information  and  advice  on  the  use  of 
threat  systems. 

•  Information  and  advice  on  test 
instrumentation  systems  and 
communications  capabilities,  costs,  and 
availability. 

Personnel  Support 

DESA  cannot  themselves  provide  personnel 
on  a  permanent  or  extended  basis  to  a  JT&E. 
However,  technical  assistance  can  be  made 
available  on  an  occasional  basis.  Additionally, 
DESA  can  assist  the  JTF  in  developing  and 
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coordinating  manpower  requirements  with  the 
Services  and  the  Civil  Service.  They  can  also 
assist  in  providing  contractual  support  from  their 
pool  of  Government  contractors. 

Accounting  and  Finance  Support 

Accounting  and  finance  expertise  is 
available  and  can  be  provided.  DESA  personnel 
are  familiar  with  the  accounting  and  finance 


procedures  and  peculiarities  of  all  the  Services 
and  can  be  helpful  to  the  JTF  in  establishing 
liaisons  with  the  respective  Service  finance 
centers.  DESA  can  also  provide  advice  on 
Service  procedures  and  periodic  reviews  of  JTF 
fiscal  situations  and  reports  to  assure  their 
accuracy.  However,  dedicated  DESA  finance 
and  accounting  personnel  are  not  available  to  any 
individual  JT&E. 
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ANNEX  G 

TEST  RANGES  AND  FACILITIES 


A.  INTRODUCTION 

The  JT&E  program  schedule  developed  by 
the  JFS  was  based  on  early  coordination  with 
OSD,  Joint  Staff,  and  the  Services  relative  to  the 
use  of  test  ranges  or  facilities.  The  JTF  must 
formalize  the  requirement  for  use  of  these 
ranges/facilities  with  the  range/facility  managers. 
The  coordination  for  actual  use  of  a  test  range  or 
facility  should  be  in  the  form  of  a  Program 
Introduction  Document  (PID).  Although  range 
and  facility  use  was  included  in  the  Consolidated 
Resource  Estimate  (CRE),  the  range/facility 
manager  will  take  no  action  on  this  requirement 
until  a  PID  is  received.  PIDs  are  completed 
using  the  instructions  in  Universal 
Documentation  System  (UDS)  Document  501-79 
and  the  supporting  agency  instructions.  The 
range/facility  manager  will  provide  the  JTF  with 
a  statement  of  capability  as  acceptance  of 
responsibilities  as  outlined  in  the  PID. 

The  PID  should  be  based  on  a 
comprehensive  survey  of  the  test  range  or  facility 
by  a  JTF  survey  team.  The  survey  data  and 
information  will  form  the  basis  for  detailed  test 
planning  and  decision  making.  The  survey  team 
should  focus  on  technical,  operational,  and 
logistic  considerations  required  for  the  support 
necessary  to  execute  the  planned  test  activity. 
Circumstances  may  be  encountered  that  require 
follow-on  visits  to  evaluate  potential  problem 
areas.  The  survey  team  should  take  the 
necessary  time  to  do  the  job  right. 

The  survey  team  should  review  the  current 
versions  of  the  following  documents  prior  to 
beginning  the  survey: 


•  DoD  Directive  3200. 11-D  Major  Range 
and  Test  Facility  Base  Summary  of 
Capabilities. 

•  Report  on  Range  Utilization  and 
Improvement  Plans. 

•  USAF  Armament  Division  Technical 
Facilities  Manual,  Volumes  I,  II,  and  III. 

•  US  Army  Test  Facilities  Register, 
Volumes  I  and  II. 

•  Test  and  Range  Facilities  Catalog  — 
Naval  Sea  Systems  Command 

•  Naval  Test  Facilities  Data  Base 
(Technet) 

The  survey  should  acquire  data  and 
information  on  the  following  capabilities  and 
should  conclude  with  well  documented 
recommendations  on  which  planners  and 
managers  can  make  decisions. 

B  .  TEST  RANGE  OR  FACILITY 

1.  Describe  the  range  or  facility  in  detail. 
Provide  the  geographical  location, 
topography,  noncontiguous  aspects,  and 
climatic  conditions  associated  with  the 
range  or  facility. 

2.  Provide  the  size  and  dimensions  of  the 
range  or  facility  and  identify  any  restricted 
land  or  air  space  usage  requirements. 
Include  maps  and  technical  documentation 
when  appropriate. 

3.  Address  the  range  or  facility  operating 
schedule  and  availability,  EMI/EMC,  and 
proximity  of  the  area  to  military  and 
commercial  flight  patterns. 
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4.  Address  the  availability  of  floor  space  in 
existing  structures  that  can  be  used  by  JTF 
personnel. 

5.  Identify  the  range  or  facility  management 
hierarchy  and  directives  for  management, 
conduct,  and  support  of  test  activities. 

C.  LOGISTICS 

1 .  Address  range  or  facility  accessibility. 

•  Air-  Identify  and  determine  the  location 
of  military  and  commercial  airfields 
servicing  the  range  or  facility. 

•  Roads-  Identify  intra-range  and  facility 
roads  to  commercial  transportation 
facilities. 

•  Water-  Identify  port  or  access 
waterways,  distances  to  the  range  or 
facility,  restrictions  to  use,  and 
suitability  for  test  activity  use. 

2.  Identify  the  nearest  military  establishment 
that  can  provide  logistic  support.  Identify 
POCs  and  determine  the  administrative 
vehicle  to  acquire  support  and  obtain 
information  on  the  availability  of  the 
following: 

•  Billets  for  officers,  enlisted,  and 
contractors. 

•  Base  Exchange. 

•  Clothing  Sales. 

•  Commissary. 

•  Concessions  (barber  shop,  laundry, 
movies,  etc.). 

•  Dining  facilities. 

•  Medical  services. 

•  POL  (to  include  delivery  to  test  sites). 

•  Reference  library  (technical). 

•  Religious  services. 

•  Secure  storage. 


•  Supply  support. 

•  Transportation  (passenger  and  freight). 

•  Security  (guard  service). 

D.  CIVIL  ENGINEERING 

1.  Determine  the  level  of  support  that  civil 

engineers  are  able  to  provide  and  the 

response  times  associated  with  the  support. 

2.  Determine  the  type  of  electrical  service  that 

is  available  at  test  sites  to  include: 

•  Prime  power  or  generator  power. 

•  Capacity  of  power  plant  to  include 
voltage,  frequency,  and  phasing. 
Determine  size  of  power  plant  and 
availability  of  spare  service. 

•  Determine  if  back  up  power  is  available 
and  wheather  it  is  manual  or  automatic 
start  up. 

•  Determine  if  grounding  and  lightning 
protection  is  available  for  range  or 
facility  systems. 

E.  SECURITY 

Determine  the  existence  of  range  or  facility 

security  plans  and  check  the  following: 

•  Area  security  provisions  and  assigned 
priorities  for  their  protection. 

•  Control  measures  and  established 
restrictions  to  access  and  movement  into 
critical  areas  (personnel,  vehicles,  and 
materials). 

•  Determine  the  availability  of  aids  to 
security,  such  as  protective  barriers, 
signs,  gates,  protective  lighting  systems, 
communications,  and  security  forces. 

•  Determine  if  contingency  plans  are 
responsive  to  emergency  situations. 


G-2 


JT&E  Handbook 
February  1, 1996 


•  Determine  the  requirements  for  passing 
clearance  data  on  personnel  for  range  or 
facility  access. 

F.  SAFETY 

Establish  contact  with  the  range  or  facility 
safety  officer  and  obtain  copies  of  safety 
directives,  brochures,  POCs,  and  key  telephone 
numbers.  Also  obtain  information  on  the 
following: 

•  Range  or  facility  speed  limits. 

•  Use  of  rental  cars. 

•  Identify  the  meanings  of  different 
colored  warning  lights. 

•  Identify  precautions  required  at  test  sites. 

•  Identify  the  criteria  for  evacuation  of 
range  or  facilities. 

•  Determine  the  locations  of  first  aid 
stations. 

•  Identify  restricted  or  hazardous 
operations  areas. 

•  Identify  the  methods  of  notification  for 
range  or  facility  closure. 

G.  RANGE  OR  FACILITY  SYSTEMS 

AND  EQUIPMENT 

Range  and  facility  systems  and  equipment 
are  normally  identified  in  master  planning 
documents.  The  JTF  survey  team  should  obtain 
copies  of  these  documents,  if  possible.  If  not, 
collect  the  data  listed  in  the  following  paragraphs. 
Identify  the  equipment  or  systems,  determine  the 
quantity  on  hand,  and  obtain  their  authorized 
frequencies,  operational  locations,  and 
geographical  coordinates.  Some  of  the 
frequencies  may  be  classified. 

•  Communications.  Determine  if  the 
required  frequencies  are  authorized. 


•  Targets.  Determine  locations,  multi- 
spectral  signatures,  and  restrictions  to 
use. 

•  Range  or  Facility  Instrumentation. 
Identify  the  data  source  mechanization, 
the  end  product,  and  fidelity  of  the  data 
source.  Determine  the  data  formats  and 
elements  for  each  source  product. 
Determine  if  the  systems  are  certified 
prior  to  the  start  of  missions  and  if 
system  modifications  are  controlled. 

(1)  Scoring  Systems. 

(2)  Acquisition  Systems. 

(3)  Threat  Systems. 

H.  RANGE  OR  FACILITY  SUPPORT 

SYSTEMS 

I .  Range  Timing  System. 

•  Identify  the  time  source,  code  format  and 
timing  technique  used. 

•  Determine  if  system  is  available  to  all  test 
sites. 

•  Determine  compatibility  with 
instrumentation  requirements. 

•  Determine  availability  of  backup 
sources. 

•  Determine  time-synch  capabilities. 

2.  Integrated  Air  Defense  Systems. 

•  Determine  what  systems  are  available. 

•  Determine  if  systems  can  function  in 
automatic,  semi-automatic,  and  manual 
modes. 

•  Determine  system  hardware 
configurations. 

3 .  Intra-base  Radios. 

•  Determine  nets  that  are  available  and 
their  respective  functions. 
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•  Determine  types  of  equipment  and 
authorized  frequencies. 

•  Determine  the  availability  of  radios  for 
white  communications. 

4 .  Range  or  Facility  Calibration  Systems. 

•  Define  system  configuration/  network. 

•  Determine  if  systems  are  integrated  with 
timing  system. 

•  Determine  if  calibration  models  and 
databases  of  historical  information  are 
available. 

5 .  Automatic  Data  Processing. 

•  Define  the  computer  and  the  associated 
data  collection  systems. 

•  Describe  the  computer  hardware 
interface  with  the  data  sources. 

•  Determine  if  results  of  test  are  available 
on  a  near-real-time  basis. 

•  Identify  and  obtain  copies  of  computer 
products. 

I.  DATA  DISPLAY  SYSTEMS 

1 .  Identify  the  type,  location,  and  function  of 

data  display  systems. 


2.  Identify  computer  programs  in  the  system, 
and  the  interfaces  that  accept  and  feed  the 
raw  data  to  the  computer. 

3 .  Determine  if  the  data  display  system  is  on  a 
near-real-time  basis.  Determine  if  the  data 
display  system  has  the  capability  to  produce 
hard  copies  of  the  displayed  data. 

J.  FINANCIAL 

Identify  the  point  of  contact  and  the 
established  procedures  for  reimbursement  of 
costs  associated  with  the  use  of  the  range  or 
facility  and  options  for  cost  sharing  that  might  be 
required. 

K.  SUMMARY 

Provide  an  overall  assessment  of  the  range 
or  facility,  the  anticipated  support,  and  details 
that  might  contribute  to  test  planning  and 
execution. 

L.  RECOMMENDATIONS 

Be  specific.  If  the  survey  team  discovers 
problems  that  could  impact  test  execution,  the 
report  should  so  state.  The  report  should  also 
document  the  findings  and  identify  the  specific 
limitations  that  use  of  the  range  or  facility  would 
have  on  test  execution. 
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ANNEX  H 

TEST  PROGRAM  COST/BUDGET  MATRIX 


Joint  Feasibility  Studies  (JFS)  are  required 
to  develop  a  Consolidated  Resource  Estimate 
(CRE)  and  a  by-year  budget  request  if 
recommending  the  chartering  of  a  Joint  Test  and 
Evaluation  (JT&E).  The  attached  test  program 
cost/budget  matrix  illustrates  how  these 
resource  requirements  can  be  translated  into 
funding  requirements.  The  matrix  attempts  to 
capture  the  primary  JTF  activities  that  have 
associated  costs  and  the  methods  that  could  be 
used  to  estimate  the  dollars  required  to  support 
these  particular  activities.  Once  the  costs  of 
individual  items  have  been  determined,  they  are 
aggregated  and  projected  into  a  yearly  budget 
estimate.  This  matrix  is  generic  in  nature  and  will 
need  to  be  adjusted  to  fit  the  specific 
requirements  of  each  JT&E.  Some  JT&Es  will 
have  unique  testing/activity  requirements  not 
reflected  by  the  topics  in  the  matrix  while  others 
will  have  no  need  for  some  of  the  topics  listed. 

In  general,  things  to  consider  in  filling  out 
the  matrix  for  a  particular  JT&E  include: 


•  Aircraft/ships/platforms  by  type/sorties. 

•  Weapon(s)  by  type. 

•  Communication  equipment  by  type  and 
mission  (operational  and  test  support). 

•  Target(s). 

•  Characterization  requirements. 

•  Instrumentation  (beacon,  pod,  TSPI, 
GPS,  TACTS,  ACMI,  GRDCUS, 
VTR,  War  Wagons,  etc.). 

•  Meteorological  requirements. 

•  Modeling/simulations. 

•  Data  sources/devices  (9  track,  floppy 
disks,  Bernoulli,  bar  code  labels,  etc.). 

•  Data  collectors  (government,  civilian, 
military,  contractors). 

•  Operational  equipment. 

•  Test  unique  equipment  (items 
consumed  during  testing  or  are  no 
longer  of  value  after  the  JT&E  is 
complete). 
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PRIMARY  ACTIVITIES 


COST  CATEGORY 


METHOD/EXAMPLE 


i  I 


DIRECT  TEST  COSTS 


TEST  PREPARATION 

Environmental  Impact 

Note:  Preparation  costs  must 
be  prepared  for  each  test 
site/event. 

Safety  Plan 

Train/Coordinate  Test 
Instrumentation  Operators 

-  number  of  operators 

-  travel  to  training  site 

-  per  diem  during  travel  and  TDY 

-  rental  cars 

Test  Communication  Network 

-  packaging  and  shipping 

-  communication  network  setup 

-  travel  to  test  site 

-  per  diem  during  travel  and  TDY 

-  rental  cars 

TEST  ASSETS 

Test  Specific  Materials 

-  smoke  generators 

-  remote  controls 

-  woodland,  snow,  desert 
camouflage  screens/supports 

-  runway  false  operating  surface 

Note:  These  are  materials 

-  decoy  buildings  and  piers 

prepared  for  each  test  site/event 

-  radar  comer  reflectors 

and  be  will  be  unique  for  each 
JTF.  Examples  are  from  JCCD. 

-  photo  image  canvas 

-  expendables  (e.g.  fog  oil, 
tonedown  paint,  RAM/IR 
coatings,  packaging  &  shipping 
costs) 

Instrumentation 

-  rental/lease  of  instruments 

-  calibration  costs 

-  packaging  and  shipping 

-  operator  costs  if  not  included 
elsewhere  (e.g.  pay  & 
allowances,  travel,  per  diem, 
rental  car) 

Projected  TDY  Costs 

-  transportation/travel  (for  each 
site/person  to/from  each  home 
station) 

-  p)er  diem  (per  person  per  day  for 
number  of  days) 

-  rental  car  cost  (number  of  cars, 
projected  days,  average  cost  per 
day) 

Test-Related  Contract  Support 

-  JTF  on-site  support 

-  test  site/event  support 

H-2 


JT&£  Handbook 
February  1, 1996 


PRIMARY  ACTIVITIES 

COST  CATEGORY 

METHOD/EXAMPLE 

TEST  ASSETS  (cont.) 

Range  Costs 

-  range-owned  instrumentation 

-  range  control/data  processing 

-  range  communication  costs 

-  range  utilization 

-  range  safety/fire/medical 
emergency  crew 

Pretest  Modeling  Costs 

-  development  of  input  parameters 
and  characteristics  to  run  the 
model 

-  model  modification  cost 

-  projected  cost  to  perform 
modeling 

Dry  Runs 

-  execution  of  unique  trials 

TEST  DATA  PROCESSING  COSTS 

DATA  COMPILATION 

Note:  Prepared  for  each  test 
site/activity.  It  includes  the 
physical  handling  of  the  data, 
input,  documenting  calibration 
procedures  and  instrumentation 
characteristics,  obtaining  data 
samples,  etc.). 

Manpower  Required  (man 
days/months) 

-  targets 

-  participants  (e.g.  aircraft,  tanks, 
ships,  etc.) 

-  meteorological 

Test  Range  Data  Retrieval 
Reduction 

DATA  REDUCTION 

AND  RE-COMPILATION 

Note:  Prepared  for  each  test 
site/event.  It  includes  data 
reduction  and  re-compilation 
processing  of  data,  checking 
range  to  target  data,  validating 
calibrations,  assimilating  data 
from  similar  instruments  at 
different  sites/events 

Manpower  Required  (man 
days/months) 

-  per  instrumentation  requirement 

DATA  ANALYSIS 

Note:  Prepared  for  each  test 
site/activity. 

Manpower  Required  (man 
days/months) 

-  per  instrumentation  or 
test/activity  requirement 

DATA  COMPARISON 

Note:  Multi-treatment/cross¬ 
platform  comparisons 

Manpower  Required  (man 
days/months) 
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PRIMARY  ACTIVITIES 

COST  CATEGORY 

DATA  ARCHIVE 
PREPARATION 

Note:  Prepared  for  each  test 
site/activity. 

Manpower  Required  (man 
days/months) 

COMPARE/ANALYZE 
PREDICTED  TO 

ACTUAL  TEST 

RESULTS 

Manpower  Required  (man 
days/months) 

ANALYZE 

COMPARATIVE  TEST 
RESULTS 

Note:  Multi-tests/activities. 

Manpower  Required  (man 
days/months) 

METHOD/EXAMPLE 


TEST^UNIQUE 


TEST-UNIQUE 

COMPUTER 

HARDWARE 


TEST-UNIQUE 
STORAGE  DEVICES 


TEST-UNIQUE  OTHER 
HARDWARE 


TEST-UNIQUE 

COMPUTER 

MAINTENANCE 


TEST-UNIQUE 

COMMERCIAL 

SOFTWARE 


COMPUTER  HARDWARE/SOPTWARE  COSTS 


-  type(s)  of  station 

-  cost/numbers 


Workstations 


High  Resolution  Window 
Terminals 


Portable  Computer  Hardware 


Tape  Drives 


Disk  Drives 


Laser  Printers 


Color  Printers 


Dot  Matrix  Printers 


Other  Printers 


Network/Hardware  Management 


-  type(s)  of  terminals 

-  cost/numbers 


-  type(s)  of  computers 

-  cost/numbers 


-  type(s)  of  server 

-  cost/numbers 


-  type(s)  of  tape  drives 

-  cost/numbers 


-  type(s)  of  disk  drives 

-  cost/numbers 


-  type(s)  of  laser  printers 

-  cost/numbers 


-  type(s)  of  color  printers 

-  cost/numbers 


-  type(s)  of  dot  matrix  printers 

-  cost/number 


-  type(s)  of  other  printers 

-  cost/number 


-  manpower  required  (man  months) 


Hardware  Maintenance 


Licensed  Commercial  Software  -  type  of  software 

-  cost/numbers  of  copies 
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PRIMARY  ACTIVITIES 


COST  CATEGORY 


METHOD/EXAMPLE 


J_ L 


DATABASE-RELATED  COMPUTER  HARDWARE/SOFTWARE  COSTS 

DATABASE-RELATED 

COMPUTER 

HARDWARE 

Workstations 

-  type  of  workstations 

-  cost/number  of  workstations 

Computer  Terminals 

-  type  of  terminals 

-  cost/number  of  terminals 

Server(s) 

-  type  of  servers 

-  cost/number  of  servers 

DATABASE-RELATED 
STORAGE  DEVICES 

Tape  Drive(s) 

-  type  of  tape  drives 

-  cost/number  of  tape  drives 

Optical  Disk(s) 

-  type  of  optical  disks 

-  cost/number  of  optical  disks 

Disk  Drive(s) 

-  type  of  disk  drives 

-  cost/number  of  disk  drives 

DATABASE-RELATED 
OTHER  HARDWARE 

Laser  Printers 

-  type(s)  of  laser  printers 

-  cost/number  of  laser  printers 

Color  Printers 

“  type(s)  of  color  printers 
-  cost/number  of  color  printers 

Dot  Matrix  Printers 

-  type(s)  of  dot  matrix  printers 

-  cost/number  of  dot  matrix  printers 

DATABASE-RELATED 

MAINTENANCE 

Hardware  Maintenance 

DATABASE-RELATED 

SOFTWARE 

Licensed  Software 

-  type  of  software 

-  cost/number  of  copies 

Purchased  Software 

-  type  of  software 

-  cost/number  of  copies 

Developed  Software 

-  type  of  software 

-  projected  cost/manpower 

Other  Software 

OTHER  CONTRACT  SUPPORT  COSTS 

TEST  DESIGN 

SUPPORT 

Manpower 

-  cost  per  man  day /month/year 

TEST  PLANNING 
SUPPORT 

Manpower 

“  cost  per  man  day/month/year 

REPORT 

PREPARATION 

Manpower 

-  cost  per  man  day/month/year 

DEVELOP/REFINE 
MODEL  CALIBRATION 
TOOLS 

Manpower 

“  cost  per  man  day/month/year 
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OTHER  COSTS 


NON-TEST  EVENT 
RELATED  TRAVEL 


Conferences/Technical 

Interchanges 

-  number  of  trips 

-  average  cost  (average  length, 

max.  per  diem,  rental  car, 
average  transportation  cost) 

Administrative  Travel 
(IPR,  TAB,  GOSC,  etc.) 

-  number  of  trips 

-  average  cost  (average  length, 

max.  per  diem,  rental  car, 
average  transportation  cost) 

LEAD  AND  PARTICIPATING  SERVICE  COSTS  (NON-OSD) 


Civilian  Pay 


Office  Facilities 


Vehicle  Rental 


TDY  and  Per  Diem  (non-JTF) 


Base  Support  (ISSA) 


Platforms  and  Equipment  -  Flying  Hours 

-POL 
-  Munitions 


Furniture 


Non-test  Specific  Supplies  and 
Equipment 
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ANNEX  I 

ENVIRONMENTAL  CONCERNS 


A.  INTRODUCTION 

JTF  planning  must  consider  the 
environmental  effects  of  test  activities  in  the 
development  of  test  plans  and  procedures. 
Failure  to  consider  environmental  compliance 
requirements  could  result  in  negative 
consequences  to  include  test  activity  cancellation 
or  unacceptable  schedule  delays  in  execution  of 
the  JT«&E.  Avoiding  these  consequences 
requires  that  the  JTD  understand  and  develop  an 
appreciation  of  the  National  Environmental 
Policy  Act  (NEPA)  and  associated  DoD 
Directives. 

It  is  important  when  planning  JT&E  test 
activities  to  remember  that  DoD  is  the  Federal 
leader  in  agency  environmental  compliance  and 
protection.  There  are  more  than  40  Federal 
environmental  statutes  that  could  affect 
contemplated  JT&E  test  activities.  State  and  local 
environmental  laws  are  derived  from  these 
Federal  regulations  and  are  usually  more 
restrictive. 

This  annex  provides  information  on 
environmental  issues  and  procedures  that  might 
apply  to  the  JT&E.  If  test  activities  under 
consideration  are  to  be  conducted  on  an 
established  DoD  facility  or  major  range  test 
facility,  coordination  with  local  environmental 
personnel  and  compliance  with  established  local 
procedures  should  be  adequate. 

Most  military  or  Government  test  facilities 
have  their  own  environmental  departments  that 
have  completed  many  environment  impact  studies 
and  subsequent  statements  based  on  previous  test 


activities.  These  environmental  impact  statements 
can  be  used  provided  the  proposed  JT&E  test 
activity  has  similar  assets  and  scenarios  of  past 
test  activities. 

If  the  JT&E  is  considering  test  activities  on 
non-DoD  lands  or  test  activities  that  will  require 
the  transport  of  equipment  or  materials  over 
public  transportation  systems,  the  JTD  must 
become  familiar  with  and  consider  the  possible 
impact  of  environmental  directives  and 
regulations  for  the  areas  of  interest.  These 
directives  and  regulations  can  be  obtained  by 
contacting  the  Environmental  Protection  Agency 
(EPA)  Regional  Offices.  The  EPA  can  also 
provide  assistance  relative  to  state  and  local 
regulations  for  the  area  of  interest. 

When  requested,  DESA  can  provide  help  to 
the  JTD  to  ensure  that  environmental  issues  are 
properly  considered.  Specific  help  includes  legal 
counseling,  access  to  environmental  planning 
tools,  databases,  and  expertise  and  advice  on 
obtaining  special  instrumentation  that  might  be 
required  to  monitor  environmental  compliance. 

B.  TEST  ACTIVITY  CONSIDERA¬ 
TIONS 

The  environmental  challenge  for  the  JTD  is 
the  identification  of  proposed  test  activity  actions 
that  might  produce  adverse  biological,  ecological, 
or  socioeconomic  effects.  Some  test  activities 
will  have  no  impact  on  these  areas.  In  others,  the 
projected  impact  might  range  from  some  to 
severe.  In  any  case,  the  planned  test  activities 
must  comply  with  environmental  laws  and 
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regulations.  Two  basic  tenants  of  these  laws  and 
regulations  are: 

•  Test  activity  procedures  must  ensure  that 
environmental  information  is  available  to 
decision  makers  and  citizens. 

•  Test  activity  plans  must  consider 
reasonable  alternatives  to  avoid  or 
minimize  adverse  environmental  effects. 

Environmental  regulations  are  classified  as 
either  procedural  or  substantive.  The  NEPA 
requires  all  Federal  agencies  to  consider 
environmental  impacts  during  the  decision 
making  process  regarding  proposed  test 
activities.  Depending  on  whether  a  proposed  test 
activity  could  significantly  affect  the 
environment,  one  of  three  levels  of  analysis  is 
required  (1)  Categorical  Exclusions  (CATEX), 
(2)  Environmental  Assessments  (EA),  and  (3) 
Environmental  Impact  Statements  (EIS). 

Categorical  Exclusions 

Categorical  Exclusions  are  those  test 
activities  that  do  not  have  a  significant  individual 
or  cumulative  effect  on  the  environment  or  have 
previously  been  found  to  have  no  such  effect. 
Categorical  Exclusion  activities  are  listed  in  DoD 
Directive  6050.1  and  they  do  not  require  an  EA 
or  EIS. 

Environmental  Assessments 

An  EA  is  an  analysis  of  the  potential 
environmental  impact  of  a  proposed  test  activity. 
An  EA  may  be  required  when  the  JTD  can  not 
determine  beforehand  whether  the  test  activity 
under  consideration  will  affect  the  environment 
or  will  be  controversial  with  respect  to 


environmental  effects.  If  required,  the  EA  will 
either  conclude  that  the  test  activity  will  not 
significantly  effect  the  environment,  thus 
resulting  in  the  preparation  of  a  Finding  of  No 
Significant  Impact  (FONSI)  or  will  conclude  that 
the  test  activity  could  have  a  significant  impact. 

Environmental  Impact  Statements 

The  JTF  must  identify  test  activities  that 
could  involve  potential  environmental  impacts. 
The  environmental  impact  of  proposed  test 
activities  must  be  considered  in  the  selection  of 
the  test  method  for  conducting  the  planned 
activities  at  specific  locations.  If  environmental 
impact  is  anticipated  and  there  is  no  alternative 
test  method,  the  JTF  may  be  required  to  conduct 
an  EIS.  The  following  are  characteristics  of  an 
EIS. 

•  An  EIS  is  a  formal  consideration  of 
environmental  consequences.  It 
addresses  the  nature  of  the  test  activity 
and  its  potential  impact  on  the 
environment.  The  EIS  must  address  all 
measures  that  will  be  taken  to  minimize 
these  impacts  and  alternatives  to  the 
proposed  test  activity,  to  include  the 
consequences  of  not  doing  the  entire 
test. 

•  An  EIS  is  both  complex  and  lengthy. 

•  The  length  and  cost  of  an  EIS  can  be 
extensive  and  may  require  several  man- 
years  of  specialized  analytical  effort  to 
complete.  Thus  the  requirement  for  the 
identification  of  a  potential  EIS 
requirement  for  proposed  JT&E  test 
activities  cannot  be  overemphasized. 

•  It  is  possible  to  have  an  EIS  prepared  by 
a  contractor. 
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C .  ENVIRONMENTAL  IMPACT 

CONSIDERATIONS 

The  JTF  should  as  a  minimum  include  the 
following  in  their  consideration  of  environmental 
impacts: 

•  Determine  test  activity  scenario, 
location,  assets,  environmental 
concerns,  and  considerations. 

•  Determine  if  test  facility  has 
environmental  impact  statements  that 
would  be  applicable  to  the  proposed  test 
activity.  The  test  facility  managers  will 
know  what  requirements  are  fulfilled  or 
need  further  study. 

•  Determine  what  environmental  impact 
actions  are  ongoing. 

•  If  environmental  actions  will  be 
required,  estimate  associated  cost,  and 
time  to  accomplish. 

•  Consider  other  test  activity  options. 

D.  TEST  AND  EVALUATION  ISSUES 

The  following  is  a  list  of  some  test  and 
evaluation  issues  that  need  to  be  considered  by 
the  JTF: 

•  Weapon  Firing  (noise  and  air 
emissions). 

•  System  maintenance  (waste  solvents  and 
oils). 

•  Vehicle  operation  (noise,  terrain 
impacts,  and  air  emissions). 

•  Contamination  of  land  resources  and 
unexplored  ordinance. 

•  Accidents  involving  hazardous 
materials. 

•  Test  site  cleanup. 

•  Land  use  management. 

•  Wildlife  protection. 

•  Archaeological  and  historical  resources. 


E.  PUBLIC  INVOLVEMENT 

Public  participation  in  preparing  EAs  and 
EISs  is  a  reality  that  the  JTF  will  have  to 
consider.  Should  a  test  activity  involve  the 
potential  for  environmental  impact,  the  JTD 
should  establish  appropriate  communications 
with  local  authorities  and  interested  parties.  In 
determining  the  extent  to  which  public 
participation  could  be  required,  the  JTD  should 
consider  the  following  factors: 

•  The  magnitude  of  the  environmental 
considerations  associated  with  the 
proposed  test  activity. 

•  The  extent  of  anticipated  public  interest. 

•  Relevant  questions  of  national  security 
and  classification. 

F .  CLASSIFIED  TEST  ACTIVITIES 

Classified  test  activities  must  comply  with 
the  same  environmental  laws,  regulations  and 
directives  as  unclassified  tests.  EISs  and  EAs 
will  have  to  be  prepared,  safeguarded,  and 
disseminated  according  to  the  requirements 
applicable  to  the  classified  information.  When 
feasible,  the  documents  should  be  organized  so 
that  classified  portions  are  included  as 
appendices,  and  the  unclassified  portions  may  be 
made  available  to  the  public. 

A  classified  EA/EIS  serves  the  same 
informed  decision  making  purpose  as  the 
unclassified  versions.  Even  though  the  classified 
EA/EIS  does  not  undergo  full  public  review  and 
comment,  it  will  be  part  of  the  information 
package  for  each  JT&E  activity.  The  content  of  a 
classified  EA/EIS  must,  therefore,  meet  the  same 
content  requirements  that  are  applicable  to  a 
published  unclassified  version. 
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G .  JTF  FOREIGN  TEST  ACTIVI  TIES 

The  environmental  situation  of  a  joint  test 
conducted  outside  the  territories  of  the  United 
States  will  depend  on  the  requirements  and 
standards  set  by  the  host  nation  and  Status  of 
Forces  Agreement  (SOFA).  Thus,  if  a  test 
activity  is  considered  at  a  non-U.S.  location,  the 
JTD  must  be  familiar  with  the  specific  regulations 
that  apply  to  testing  in  the  host  country  and  the 
specific  locale  within  that  country.  All  U.S. 


facilities  located  outside  the  territories  of  the 
United  States  are  required  to  conduct  their  test 
activities  in  an  environmentally  safe  manner  and 
in  compliance  with  both  host  country  and  U.S. 
regulations  relating  to  environmental,  natural 
resource  protection,  and  occupational  health  and 
safety.  The  two  major  U.S.  directives  that  apply 
to  or  specifically  address  testing  issues  are: 

•  DoD  Directive  4120.14. 

•  EO  12114:  Environmental  Effects 
Abroad  of  Major  Federal  Actions. 
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ANNEX  J 

SECURITY  CLASSIFICATION  GUIDE 


If  the  JT&E  involves  the  development  of  an  SCG  format  developed  by  a  recent  JTF  that 
classified  material,  the  JTD  should  consider  the  could  be  tailored  to  the  specific  requirements  of  a 
requirement  to  develop  a  Security  Classification  JT&E. 

Guide  (SCG).  The  following  is  an  example  of 


A.  COVER 
Title: 


Issued  By: 


Approved  By: 


Date: 


Action  Officer: 


Distribution  Notice:  This  document  covers  the  operational  use  of  military  systems  and 

hardware.  Distribution  restricted  to  DoD  and  DoD  contractors. 
Further  dissemination  only  as  directed  by  JXXX/JTD,  XX  Road, 
Any  Town,  USA. 


B.  SECTION  I 


GENERAL  INSTRUCTIONS 

1 .  PURPOSE.  To  provide  instructions  and  guidance  on  the  classification  of  information 
developed  by  the  JXXX/JTF. 
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2 .  AUTHORITY 


a.  This  guide  is  issued  under  the  provisions  of  Executive  Order  12356;  DoD  5200. IR, 
Information  Security  Program  Regulation;  DoD  5220.22M;  and  DoD  5220.22R. 

b .  This  guide  constitutes  the  authority  for,  and  may  be  cited  as  the  basis  for  classification, 
regrading,  or  declassification  of  information  and  material  originated  by  the  JXXX/JTF. 

3 .  OFFICE  OF  PRIMARY  RESPONSIBILITY  (OPR).  All  inquiries  concerning  the 
content  and  interpretation  of  this  guide  should  be  addressed  to  JXXX/JTD,  XX  Road,  Any 
Town,  USA  XXXXX-XXXX. 

4 .  CLASSIFICATION  RECOMMENDATIONS.  Recommended  changes  will  be 
completely  documented  and  justified.  Recommendations  should  be  made  through 
appropriate  channels  to  the  OPR.  Pending  final  decision,  contested  classified  information 
shall  be  handled  and  protected  at  the  higher  classification.  All  users  are  encouraged  to  assist 
in  improving  and  maintaining  the  currency  and  adequacy  of  this  guide. 

Because  of  interface  relationships  of  program  material,  unclassified  items  of  information  may 
become  classified  or  sensitive  when  combined  into  a  single  document.  Therefore,  care  must 
be  taken  prior  to  the  release  of  any  related  information,  even  though  it  is  unclassified. 
Further,  the  release  of  information  should  be  based  upon  the  recipient's  strict  need-to-know. 

5 .  APPLICATION,  REPRODUCTION,  AND  DISSEMINATION.  This  guide  is 
intended  specifically  for  the  use  and  guidance  of  individuals  with  a  need-to-know 
requirement  for  information  contained  herein.  Reproduction  and  dissemination  will  be  at  the 
discretion  of  the  OPR. 

6 .  PUBLIC  RELEASE.  The  fact  that  certain  information  is  shown  to  be  unclassified  does 
not  authorize  public  release.  Public  release  of  unclassified  information  must  be  approved  by 
the  JTD. 

7 .  FOREIGN  DISCLOSURE.  Release  of  program  information  or  hardware/software  is  not 
authorized  to  any  foreign  entity  or  foreign  national  without  prior  approval  by  the  JXXX/JTD. 
All  proposed  releases  of  program  information  to  any  foreign  government  shall  be  forwarded 
to  the  OPR  for  review  and  further  processing  action. 

8  .  PROGRAM  DEFINITIONS.  The  following  is  a  list  of  abbreviations  and  designations 
used  throughout  this  guide; 


a. 

C 

Confidential  Classification  Level 

b. 

COMSEC 

Communications  Security 

c. 

DoD 

Department  of  Defense 
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d.  FOUO 

For  Official  Use  Only 

e.  FSD 

Feasibility  Study  Director 

f.  JXXX 

Joint  XXX  Feasibility  Study 

g.  JFS 

Joint  Feasibility  Study 

h.  JTD 

Joint  Test  Director 

i.  JTF 

Joint  Test  Force 

j,  JT&E 

Joint  Test  &  Evaluation 

k.  MACOM 

Major  Command 

1.  NOFORN 

Not  Releasable  to  Foreign  Nationals 

m.  OADR 

Originating  Agency  Determination  Required 

n.  OPR 

Office  Of  Primary  Responsibility 

0.  OPSEC 

Operations  Security 

p.  R&D 

Research  and  Development 

q.  RCS 

Radar  Cross  Section 

r.  RDT&E 

Research,  Development,  Test  &  Evaluation 

s.  S 

Secret  Classification  Level 

t.  SAM 

Surface-to-Air  Missiles 

u.  SCG 

Security  Classification  Guide 

V.  T&E 

Test  And  Evaluation 

w.  TS 

Top  Secret  Classification  Level 

X.  TSPI 

Time  Space  Positioning  Information 

y.  U 

Unclassified 

z.  WNINTEL 

Warning  Sensitive  Intelligence  Sources  and  Methods  Involved 

C .  SECTION  II 

OVERALL  EFFORT 

9  .  GOALS,  MISSION,  &  PURPOSE.  To  provide  instructions  and  guidance  for  the 
security  classification  of  information  and  material  originated  by  the  JXXX/JTF. 

a .  The  purpose  of  the  JXXX/JT&E  is  to  resolve  the  program  issues  (as  appropriate). 

b .  The  JXXX/JTF  Security  Classification  Guide  (SCG)  is  designed  to  protect  information 
related  to  (e.g.,  U.S.  missiles,  manned  aircraft  systems,  sensor  performance  and 
vulnerabilities,  intelligence  assets,  tactics  and  weapons  employment,  CCD,  etc.),  data 
collected  or  derived  from  other  sources  for  JXXX  use  and  various  other  aspects  of  the 
test  procedures. 

1 0 .  END  ITEM.  The  end  items  of  this  effort  include  a  JT&E  final  report. 

The  JT&E  final  report  will  normally  be  unclassified  unless  the  subject  matter  is  otherwise 
classified  or  so  designated  by  the  authority  or  agency  supplying  the  proposed  equipment 
and/or  techniques  to  be  tested.  Test  data  and  results  that  indicate  a  capability,  vulnerability. 
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weaknesses  or  that  would  compromise  system  survivability  will  be  classified  Secret.  Actual 
use  of  equipment  or  concepts  to  be  tested  during  test  activities,  will  be  classified  in 
accordance  with  the  guidance  of  the  JTD  or  facility  commander. 

Source  classification  guidance  must  be  consulted  when  information  and  hardware  that  is 
classified  by  other  agencies  are  utilized.  The  following  subparagraphs  contain  detailed 
classification  guidance. 

TOPIC  CLASS  DECLASS  COMMENT 

1.  GENERAL: 

a.  Information  revealing  the  U 

existence  of  general  (XXX) 

policy,  doctrine,  tactics, 
material,  techniques,  and 
training  program 

b.  Acknowledgment  that  new  U 

(XXX)  capabilities  are 
continually  being  developed 

c.  The  identity  of  U  Protect  as  FOUO 

organizational  (XXX) 

planners 

d.  Information  that  reveals  U  Note  2 

there  is  a  policy  of 

employing  peacetime 
(XXX)  to  support  security 
operations 

e.  Information  that  reveals  U  Protect  as  FOUO 

there  is  a  policy  of 

employing  (XXX)  to 
enhance  defense  of  forces 
and/or  support/conduct 
combat  operations 

f .  Information  that  reveals  a  U  Notes  2  &  3 

policy  concerning  the 

planning  and  conduct  of 
combined  (XXX)  operations 
and  exercises 
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g.  Generic  tactics  and  U 

techniques  such  as  feints, 
diversions,  ruses, 
demonstrations,  displays, 
electronic  deceptions,  and 
communications 

h.  Effectiveness  of  actual  U 

tactics  or  techniques,  if 

known,  that  are  not 
influenced  by  foreign 
knowledge 

i.  Actual  tactics  or  techniques  S 

that,  if  known,  would  aid  an 
adversary  in  the  detection  of 

tactical  (XXX)  activity  or 
facilitate  development  of 
countermeasures 


j .  Actual  tactics  or  techniques  See  Comments 
that,  if  disclosed,  might 

reveal  intelligence  sources  or 
collection  capabilities 

k.  Information  detailing  the  S 

comprehensive  capabilities 

and  goals  (current  and 
projected)  of  a  Service, 

MACOM,  base  or  unit 

l.  Information  identifying  S 

vulnerabilities  or 

deficiencies  in  tactical 
(XXX)  operational  plans, 
conceptual  plans, 
operations,  capabilities, 
tactics,  training  or 
equipment  that  would  negate 
their  effect  if  known 


OADR 


OADR 


OADR 


Note  3 


Notes  4  &  9 


Note  3 
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m.  Information  revealing  the  S  OADR  Note  3 

actual  employment  of 
(XXX)  equipment  and 
techniques  when  knowledge 
of  the  information  would  aid 
threat  planning  to  negate, 
nullify  or  counter  the  use  of 
the  means;  or,  when  the 
existence  of  a  classified 
capability  would  be  revealed 

12.  (XXX)  EQUIPMENT 
AND  TECHNIQUES 

a.  Permanent  (XXX)  U  Note  2 

equipment/techniques  used 

and/or  employed 

b.  Plans  for  specific  use  of  S  OADR 

(XXX)  that  are  deployed 

only  during  certain  stages  of 
alert  or  actual  combat 

c.  Instructions  revealing  how  U 

to  use  (XXX)  equipment 

and  techniques  (unclassified 
unless  otherwise  noted) 

13.  TEST  DATA: 

a.  Test  plans,  methodology,  U 

test  data,  or  results  that  do 

not  reveal  specific 
vulnerabilities,  capabilities, 
or  weaknesses 

b.  Test  plans,  methodology,  S  OADR 

test  data,  or  results  that 

reveal  specific 
vulnerabilities,  capabilities, 
or  weaknesses 

c.  Test  philosophy  & 
objectives 

•  If  capabilities,  S  OADR 

vulnerabilities,  or 
limitations  are  revealed 
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•  If  capabilities, 
vulnerabilities,  or 
limitations  are  not 
revealed 

d.  Test  results 

•  If  capabilities,  S  OADR 

vulnerabilities,  or 

limitations  are  revealed 

•  If  capabilities,  U 

vulnerabilities,  or 

limitations  are  not 
revealed 

e.  Actual  test  dates  U  Protect  as  FOUO 

f .  Overall  test  schedule  U  Protect  as  FOUO 

g.  Location  of  test  sites  U 

14.  DATA  ACQUISITION, 

REDUCTION,  AND 
PRESENTATION: 

a.  Mathematical  algorithms  or  U 

computer  software  routines 
that  are  used  to  reduce, 
analyze,  or  present  data 

b.  Mathematical  algorithms  or  U 

computer  software  routines 
that  are  used  to  evaluate 
(XXX)  equipment  or 
techniques  effectiveness 


Note  6  (Information 
may  be  protected  as 
proprietary) 

Note  6  (Information 
may  be  protected  as 
proprietary) 
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OADR 


Notes  7  &  8 


c.  Remotely  sensed  signatures,  S 

measurements,  reports, 

papers,  software,  synthetic 
or  enhanced  imagery, 
drawings,  etc.,  that  provide 
research  and  development 
results,  conclusions,  or 
recommendations  that  reveal 
an  enhancement  or 
vulnerability  that  could 
compromise  the 
survivability  or  operability 
of  a  high-value  asset  target, 
facility  or  installation 

d.  Specific  conclusions  and  S 

recommendations  regarding 

(XXX)  test  and  evaluation 
drawn  from  raw  or  tabulated 
data  as  described  in  (c)  and 
processed  data  as  previously 
described  if  they  reveal 
vulnerabilities  or  impact  on 
survivability 

e.  Reports  resulting  from  use  See  Comments 
of  classified  data  and/or 

analysis,  conclusions,  and 
recommendations 

f .  Compiled  database  (in  any  S 

form)  of  test  results  when 
capabilities,  vulnerabilities, 
limitations,  or  weaknesses 

are  revealed 

g.  Compiled  database  (in  any  See  Comments 
form)  of  test  results  when 

capabilities,  vulnerabilities, 
limitations,  or  weaknesses 
are  not  revealed 


OADR 


OADR 


Note  11 


Notes  6  &  8 


Note  8/Protect  as 
FOUO 
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D.  SECTION  III 


PERFORMANCE  AND  CAPABILITIES 
TOPIC  CLASS  DECLASS  COMMENT 

15.  TEST  &  EVALUATION 

a.  Information  related  to  the  S  OADR 

performance  and/or 

capabilities  of  specific 
(XXX)  equipment  or 
techniques 

b.  Information  related  to  S  OADR  Note  3 

countermeasures  that  would 

affect  the  performance 
and/or  capabilities  of  (XXX) 
equipment  or  techniques 

c.  Information  regarding  S  OADR 

current  or  projected 

performance  and/or 
capabilities  of  (XXX) 
equipment  or  techniques 

d.  Design,  performance  and/or  See  Comments  Note  9 

functional  characteristics  of 

operational  (XXX) 
equipment 

e.  Test  procedures  information 

•  If  operational  S  OADR  Note  9 

employment  tactics, 

capabilities,  or 
limitations  are  revealed 

•  If  operational  U 

employment  tactics, 
capabilities,  or 

limitations  are  not 
revealed 
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16.  DATA  REDUCTION  & 

PRESENTATION: 

a.  Test  data  and  reports  that  See  Comments 
disclose  (XXX) 

performance 

b.  Test  data  and  reports  that  do  U 

not  disclose  (XXX) 

performance 

c.  Test  data  analysis,  results 
and  conclusions 

•  If  capabilities,  S  OADR 

limitations,  or 

vulnerabilities  of  a 
system  are  revealed 

•  If  capabilities,  U 

limitations,  or 

vulnerabilities  of  a 
system  are  not  revealed 

d.  Flight  test  data/encryption  See  Comments 
codes 

e.  Raw  or  tabulated  data  U 

resulting  from  tests,  such  as 
uncaptioned  photographs, 
thermographs,  temperature 
measurements,  radar  cross- 

section  measurements,  etc. 

f .  Processed  data  from  tests,  S  OADR 

such  as  captioned 

photographs  or 
thermographs,  tabulated 
temperature  measurements, 

RCS  measurements,  etc. 

g.  Mission  tapes  obtained  from  See  Comments 
test  aircraft 

h .  Raw  test  data  tapes  and  logs 
generated  by  JXXX 


Note  9 


Note  3 


Notes  4  &  9 

Notes  7  &  9 
(Protect  as  FOUO) 

Notes  4  &  10 

Note  9  (See  Sec.  20 
a&b) 
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•  If  results  are  revealed 
which  indicate 
capabilities, 
vulnerabilities,  or 
weaknesses 

•  If  results  are  not  U 

revealed  which  indicate 
capabilities, 

vulnerabilities,  or 
weaknesses 

i.  Measurement  parameters  See  Comments  Notes4&9 

identified  on  data  sets, 

imagery,  and  individual 
target  RCS  tables 

j .  Display  of  mission  route  or  U 

maps  containing  information 
revealing  location,  scale  of 

distance  , 

17.  (XXX)  DESIGN 
INFORMATION; 

a.  If  information  concerning  S  OADR  Notes  3  &  9 

limitations,  deficiencies, 

vulnerabilities,  or 
weaknesses  that  could 
impact  on  system 
survivability  is  revealed 

b.  If  information  concerning  U 

limitations,  deficiencies, 
vulnerabilities,  or 

weaknesses  that  could 
impact  on  system 
survivability  is  not  revealed 
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E.  SECTION  IV 


SPECIFICATIONS 


TOPIC  CLASS  DECLASS 

18.  SYSTEM  AND  SUB¬ 
SYSTEM  SPECIFICA¬ 
TIONS: 

a.  T&E  activities  and  end 
products  that  reveal: 

•  Specifications  of  (XXX)  U 

equipment  or  techniques 

that  describe  items  but 
do  not  provide 
information  regarding 
signature  alteration, 
radar  cross-sections, 
etc/as  appropriate 

•  Specifications  of  (XXX)  S  OADR 

equipment  or  techniques 

that  describe  items  and 
provide  information 
regarding  signature 
alteration,  radar  cross- 
sections,  etc./as 
appropriate 

F.  SECTION  V 

CRITICAL  ELEMENTS 

TOPIC  CLASS  DECLASS 

19.  TEST  &  EVALUATION 

a.  Activities  that  reveal  test 

•  The  fact  that  equipment  S  OADR 

and  techniques  have  the 

capability  to  alter  or 
simulate  signatures  in 
any  spectral  band 
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COMMENT 


Note  9 


COMMENT 
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•  Those  features  of  S  OADR 

equipment  and 
techniques  that  provide 
levels  of  signature 
alteration  or  simulation 


b .  The  fact  that  equipment  and 
techniques  cause 
degradation  in  enemy  target 
detection  and  recognition 
capability 

U 

c.  Quantitative  data  on  actual 
degradation  in  detection  and 
recognition  range  of  target 
acquisition  systems 

S 

OADR 

d .  Conclusions  regarding  the 
current  and/or  projected 
impact  on  target  detection 
and  recognition  capability 

S 

OADR 

e.  New  principles,  procedures, 
or  doctrine  that  reveals  a 
break-through  with 
significant  military  impact 

s 

OADR 

f .  Operational  concepts  for 
implementation  of  CCD 
equipment  and  techniques 

S 

OADR 

.  DATA  ACQUISITION: 

a.  Time  Space  Positioning 
Information  (TSPI) 

See  Comments 

Note  9 

b .  HUD  systems  data 

See  Comments 

Note  9 

21.  AIRBORNE  SENSORS 
AND  INTELLIGENCE 
ASSETS: 
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G.  SECTION  VI 

VULNERABILITIES  AND  WEAKNESSES 

TOPIC  CLASS  DECLASS  COMMENT 

22.  (XXX)  EQUIPMENT  & 

TECHNIQUES: 


a.  Instructions  on  how  to  (as 
appropriate,  i.e.  operate, 
assemble,  erect,  move  or 
place)  (XXX)  equipment  or 
when  a  vulnerability  or 
weakness  is  indicated 

S 

OADR 

b.  Current  or  projected  (XXX) 
equipment  and  techniques 
that  reveal  vulnerabilities  or 

weaknesses 

S 

OADR 

c .  Information  resulting  from 
tests  regarding  current  or 

S 

OADR 

Note  3 

projected  (XXX)  equipment 
or  techniques,  deployment 
concepts,  or  counter¬ 
countermeasures  that  reveal 
an  enhancement  or 
vulnerability 

23.  DATA  REDUCTION: 

a.  Mathematical  techniques  and 
computer  codes  that  are 
related  to  assessment  of 
(XXX)  vulnerabilities  and 
weaknesses: 

•  Aid  in  the  solution,  but 
do  not  predict  signatures 
from  physical  data 
inputs 

•  Predict  signatures  from 
physical  data  inputs  if 
vulnerabilities  are 
revealed 


U  Note  8 


S  OADR  Note  9 
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•  Include  optimization  S  OADR 

routines  for  defining 
configuration  shapes  for 
low  or  minimal 
signatures 

24.  SUPPORT  EQUIPMENT; 


a.  Aircraft 

See  Comments 

Note  9 

b.  Ground  equipment 

See  Comments 

Note  9 

.  SECTION  VII 

TOPIC 

HARDWARE 

CLASS  DECLASS 

COMMENT 

25.  HARDWARE: 

a.  Hardware  nomenclature 

U 

b.  Federal  stock  number 

u 

c.  Serial  numbers 

u 

d.  Model  designation 

u 

e.  (XXX)  End  item(s)  (as 

See  Comments 

Note  5 

appropriate) 

f .  Support  aircraft  (as 

See  Comments 

Notes  5  &  9 

appropriate) 

g.  Ground  support  equipment 

See  Comments 

Notes  5  &  9 

(as  appropriate) 

26.  DATA  PROCESSING 

EQUIPMENT: 

a.  Specification/performance 

See  Comments 

Notes  1,  5  &11 

characteristics  equipment  or 

(Information  may 

instrumentation  used  to  supplier  proprietary) 

collect,  reduce,  analyze,  or 
present  data  or  used  to 
evaluate  (XXX) 


JT&E  Handbook 
February  1, 1996 


J- 


I.  SECTION  VIII 


ADMINISTRATIVE  DATA 


TOPIC 

CLASS 

DECLASS 

COMMENT 

27.  GENERAL: 

a.  (XXX)  equipment  and 
techniques  except  for  those 
hardware  and  data  items 
including  signature 
measurements, 
photographs,  drawings,  and 
specifications 

U 

Note  5 

b.  Specialized  (XXX) 

equipment  and  techniques 
for  critical  elements 

s 

OADR 

c.  Orders  to  install,  erect,  or 
emplace  individual  items  of 
(XXX)  equipment 

•  Pertaining  to  wartime 
measures  that  would 
indicate  preparations 
against  or  for  attack 

s 

OADR 

•  If  items  do  not  pertain  to 
wartime  measures  that 
would  indicate 
preparations  against  or 
for  attack 

u 

d.  Names  and  locations  of 
(XXX)  test  program 
participants 

u 

Protect  as  FOUO 

28.  DEGREEOF 

PROTECTION  IN 

TRANSIT/STORAGE/ 

PACKAGING: 

a.  When  the  information 

s 

OADR 

Note  12 

reveals  an  operational 
deficiency/limitation 


J-16 


JT&E  Handbook 
February  1, 1996 


b.  When  the  information  S 

reveals  a  classified  plan  or 
operation 

c.  When  the  information  does  U 

not  apply  to  (a)  or  (b) 

29.  T&E  PROGRAM: 

a.  Funding  (individual  U 

contracts  or  projects, 

program,  total  T&E  effort) 

b.  Budget  year,  prior  year(s),  U 

future  fiscal  year(s),  or  total 

dollars 

c.  Quantities  by  budget  or  U 


future  fiscal  year(s) 

30.  PRODUCTION  AND 
PROCUREMENT: 

a.  Contract  delivery  schedules  U 

of  (XXX)  equipment  and 

material 

b.  Delivery  schedules  for  S 

equipment  and  material 

needed  in  conduct  of  a 
test/project  when 
information  reveals 
operational 

deficiency/limitation  or 
anticipated  fielding  plan  for 
urgently  needed  or  new 
technology  system(s) 


OADR 


OADR 


Note  12 


Protect  as  FOUO 
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J.  SECTION  IX 


NOTES 


1 .  Security  classification  must  be  determined 
by  detailed  considerations  of  all  applicable 
elements.  When  partial  information  is 
presented,  the  lowest  classification 
consistent  with  adequate  protection  will  be 
assigned. 

2.  The  establishment  of  security  classification 
levels  for  the  deployment  data  within  a 
theater  is  the  responsibility  of  the  Theater 
Commander. 

3.  Any  information  revealing  possible 
countermeasures  and  counter¬ 
countermeasures  to  U.S.  military  (XXX) 
equipment/systems  will  normally  be 
classified  SECRET.  Any  detailed 
discussion,  report  or  study  of  system 
capabilities  and/or  limitations  from  which 
conclusions  may  be  drawn  that  would 
suggest  enemy  tactics  most  likely  to  defeat 
U.S.  equipment/systems  and  techniques 
will  be  classified  SECRET. 

4.  Classify  in  accordance  with  source  material 
or  information  revealed. 

5.  Equipment  shall  be  safeguarded  in 
accordance  with  the  requirements  for  the 
category  of  information  revealed  by  the 
equipment.  Commercial  equipment  is 
normally  unclassified.  Specifically 
designed  equipment  and/or  specifically 
designed  equipment  utilizing  commercial 
equipment  will  be  assigned  a  security 
classification  commensurate  with  the 


performance  characteristics  or  parameters 
of  information  that  is  revealed. 

6.  Security  classification  of  computer 
programs,  or  segments  thereof,  shall  be 
based  on  the  classification  of  the 
information  or  parameters  that  can  be 
revealed  by  the  program.  In  order  to 
accurately  classify  a  particular  program, 
review  applicable  SCGs  and  apply  only  that 
level  of  classification  required  to  protect  the 
parameters  contained  in  the  program.  This 
note  applies  to  interim  and  final  products 
(tape,  card  deck,  listings,  etc.)  as  well  as 
any  software  documentation  that  can  reveal 
the  classified  data. 

7.  Reports,  publications,  drawings, 
schematics,  photographs,  models, 
mockups,  training  aids,  and  graphs  will  be 
assigned  a  security  classification  as 
determined  by  the  appropriate  application  of 
this  security  classification  guide  and 
information  revealed. 

8 .  Unclassified  items  may  become  sensitive  or 
classified  when  compiled.  See  individual 
sections  for  overall  highest  classification. 

9.  Consult  appropriate  classification 
guidelines  that  are  applicable  to  unique 
U.S.  Military  aircraft,  ground  support 
equipment  and  systems,  collection  systems, 
facilities,  computer  programs,  and 
operational  tactics. 
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10.  The  relationship  of  data  to  telemetry, 
channel,  calibration,  raw  telemetry,  and 
frequency  of  operation  shall  be  classified  in 
accordance  with  the  information  that  might 
be  discernible  from  them.  The  telemetry 
encryption  shall  be  SECRET  and  raw 
telemetry  shall  be  unclassified.  The 
relationship  of  raw  telemetry  to  calibration 
is  classified  by  the  data  revealed. 


11.  Classify  end  item  in  accordance  with  the 
highest  classified  item  revealed. 

12.  Movement  and  storage  of  classified 
hardware  between  installation  and  activities 
will  be  accomplished  in  accordance  with  the 
provisions  of  DoD  5200. IR  and  DoD 
5220.22M. 
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ANNEX  K 

MILESTONE  REVIEWS 


A.  INTRODUCTION 

The  JTD  should  coordinate  with 
DDT&E  to  review  the  progress  of  the  joint  test 
at  the  following  milestones;  when  the  JTF  has 
been  established  and  organized  and  the 
program  issues  have  been  verified;  annual 
reviews  of  JT&E  progress  and  results;  and 
after  Service  coordination  of  the  Joint  Test  final 
report.  This  annex  provides  guidance  for  these 
reviews  in  the  form  of  examples  of  outlines  for 
the  milestone  presentations.  These  examples  of 
outlines  should  be  tailored  to  the  specific 
requirements  to  the  JTF. 

The  first  milestone  review  should  be  an 
In  Progress  Review  (IPR)  conducted  by 
DDT&E  after  the  JTF  is  organized  and  the 


program  issues  have  been  reviewed  and 
validated  as  current  and  accurate.  The  second 
and  third  milestones  should  be  annual  TAB 
reviews  of  JT&E  progress  and  results.  The 
focus  of  these  reviews  should  be  on  the 
technical  aspects  of  the  JT&E  in  terms  of 
credibility  and  risks  relative  to  resolution  of  the 
program  issues.  A  final  suggested  milestone 
would  be  a  TAB/DDT&E  review  of  the  JTF 
final  report.  This  review  should  be  conducted 
after  the  final  report  has  been  coordinated  with 
all  interested  agencies.  The  focus  of  this  final 
review  should  be  the  technical  competency  and 
credibility  of  the  JTF  conclusions  and 
recommendations.  Examples  of  outlines  for 
these  reviews  are  provided  in  the  following 
sections. 


B.  AN  EXAMPLE  OF  AN  OUTLINE  FOR  THE  FIRST  MILESTONE  REVIEW 
(IPR) 

1 . 0  Introduction 

1.1  Title/Nomination  Organization/Agency 

1 .2  Lead  Service/Executive  Agency/JTD 

1 .3  Participating  Services/Executive  Agencies 

1 .4  Other  Major  Players 

1.5  Purpose  and  Focus  of  Review  (establishment  of  JTF  and  verification  of  the  issues) 

2.0  Background 

2.1  Problem  Statement 

2.1.1  Assumptions 

2.1.2  Constraints  and  Limitations 

2.2  Joint  Test 

2.2.1  Purpose 

2.2.2  Test  Issues  and  Objectives 


K-l 


JT&E  Handbook 
February  1, 1996 


2.2.3  Test  Articles 

2.3  Scope  of  Joint  Test 

2.3.1  Mission  Requirements 

2.3.2  System  Requirements 

2.3.3  Participants 

2.3.4  Users  of  Test  Results 

3.0  JTF  Establishment  and  Organization 

3.1  Facilities 

3.2  Staffing 

3.2.1  Identify/Ouantify 

3.2.2  Formal  Personnel  Request  to  Service  POC 

3.2.3  Organization/Responsibilities 

3.3  Communications  and  ADP 

3.4  Funds 

3.4.1  Original  Budget  Estimate 

3.4.2  Revised  Budget  Estimate 

3.5  JTF  Schedule  and  WBS 

3.5.1  Original  Schedule 

3.5.2  Revised  Schedule 

3.6  Security 

3.6.1  Procedures 

3.6.2  Security  Classification  Guide 

3.6.3  SCIF  Requirements 

3.7  Coordination  Chain 

3.7.1  POCs 

3.7.2  Signature  Authority 

4.0  Program  Issues 

4.1  JT  Problem  Statement  (as  refined  by  JTD) 

4.2  Issue  Justification  Statement 

4.3  Coordination 

4.3.1  Organizations 

4.3.2  Recommended  Revisions 

4.3.3  Rectification  of  Recommendations 

4.4  Verification  of  Issues 

5.0  Problems 

6.0  Summary  of  JTF  Status 
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C .  EXAMPLE  OF  AN  OUTLINE  FOR  THE  ANNUAL  MILESTONE  REVIEW  OF 
JT&E  ACTIVITY  RESULTS 

1.0  Introduction 

1 . 1  Background  and  Relation  of  Test  Activities  to  Program  Issues 

1 .2  Purpose  of  Test  Activity 

1.3  Program  Issues  that  were  Addressed 

1 .4  Scope  of  Test  Activities 

1.4.1  Assumptions 

1.4.2  Constraints  and  Limitations 

1.5  Test  Activity  Scenarios 

2.0  Description  of  Test  Activities 

2 . 1  Overview  of  Test  Activities 

2.1.1  Location,  Dates,  and  Durations 

2.1.2  Test  Articles,  Tactics,  Procedures,  and  Concepts 

2.1.3  Participants 

2.1.4  Resource  Requirements 

2.1.5  Instrumentation 

2.2  Schedule 

2 . 3  Control  of  Test  Activities 

3.0  Joint  Test  Results 

3 . 1  Compliance  with  Test  Activity  Plans 

3.2  Actual  Data  Collected  (as  compared  to  planned) 

3 . 3  Completeness  and  Accuracy  of  Data  Collected 

3 . 4  Expected  Results 

3.5  Actual  Results 

3.6  Unexpected  Results 

3.7  Other  Accomplishments 

4.0  Problems  and  Solutions 

4. 1  Urgency  of  Need  for  Test  Results 

4.2  Systemic  Problems  and  Solutions  Employed 

4.3  Unique  or  New  Problems  and  Solutions  Employed 

5.0  Conclusions  and  Recommendations 

5.1  Findings 

5.2  Conclusions 

5.3  Recommendations 
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D .  EXAMPLE  OF  AN  OUTLINE  FOR  FINAL  MILESTONE  REVIEW  -  JTF  FINAL 
REPORT 

1 . 0  Introduction 

1.1  Background  and  Discussion  of  Problem 

1.2  Purpose  of  JT&E 

1.3  Program  Issues 

1.4  Program  Scope 

1.4.1  Assumptions 

1.4.2  Constraints  and  Limitations 

1.5  Program  Scenario 

1.6  Summary 

2.0  Joint  Test  Description 

2.1  Overview  of  JT&E 

2.2  Program  Issues  and  Objectives 

2.2.1  Coordination 

2.2.2  Test  Measures 

2.2.3  Approach  for  Resolving  Issues 

2.2.4  Test  Methods 

2.2.5  Test  Cells 

2.2.6  Overview  of  Test  Activities 

2.2.6.1  Test  Activity  1 

•  Location 

•  Resource  Requirements 

•  Instrumentation  Requirements 

•  Environmental  Considerations 

2.2.6.2  Test  Activity  n 

2.3  Test  Activity  Schedule 

3.0  Joint  Test  Results 

3 . 1  Compliance  with  Program  Test  Design 

3 . 2  Completeness  and  Accuracy  of  Data  Collected 

3 . 3  Expected  Results  (Issues  that  were  Expected  to  be  Resolved) 

3 . 4  Actual  Results  (Issues  that  were  Resolved) 

3.5  Issues  that  can  be  Resolved  by  Others  using  Program  Data 

3.6  Unexpected  Results 

3.7  Other  Accomplishments 

3.8  Test  Activity  Products 

4.0  Problems  and  Solutions 

4.1  Urgency  of  Need  for  Test  Results 

4.2  Systemic  Problems  and  Solutions 
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4.3  Unique  or  New  Problems  and  Solutions 

5.0  Findings,  Conclusions,  and  Recommendations 

5.1  OSD,  Joint  Staff,  and  Service  Concurrence  (Sensitivities  or  non  concurrences) 

5.2  Benefits  of  Participation 

5.3  Findings 

5.4  Conclusions 

5.5  Recommendations 

5 . 6  JTF  Produced  Products 

5 . 7  Institutionalization  of  Products 

5.7.1  What,  Who,  Where,  When,  and  How 

5.7.2  Oversight  of  Institutionalization  actions 

Annexes  (as  required  to  substantiate  conclusions  and  recommendations) 
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ANNEX  L 

SUGGESTED  REPORT  OUTLINES  PTP,  DMPA,  JT&E  FINAL 
REPORT,  AND  JTF  MANAGEMENT  REPORT 


A.  INTRODUCTION 

The  following  outlines  for  plans  and 
reports  are  provided  as  examples  of  some  used 
by  previous  JTFs.  If  used,  the  JTF  should  tailor 


these  formats  to  the  specific  requirements  of  their 
joint  test.  In  general,  use  of  these  or  similar 
outlines  could  benefit  the  JTF  as  DDT&E  and  the 
TAB  are  familiar  with  the  formats  used  by 
previous  joint  tests. 


B .  EXAMPLE  OF  AN  OUTLINE  FOR  A  PROGRAM  TEST  PLAN  (IF 
REQUIRED) 

This  document  should  be  an  extension  of  the  APA  and  should  not  include  repetitive 
discussions  of  information  or  data  adequately  covered  in  the  APA. 

1 . 0  Introduction 

1 . 1  Background  and  Discussion  of  Problem 

1.2  J oint  Feasibility  Study 

1.2.1  Overview  of  APA 

1.2.2  JFS  Conclusions 

1.3  Purpose  of  JT&E 

1.4  Program  Issues 

1 .5  Program  Scojie/Assumptions/Limitations 

1.6  Program  Scenario 

2 . 0  Program  Description/Methodology 

2 . 1  Overview  of  JT&E 

2.2  Program  Issues 

2.2.1  Coordination 

2.2.2  Test  Measures 

2.2.3  Approach  for  Resolving  Issues 

2.2.4  Test  Methods 

2.2.4. 1  Cost/Credibility  of  Results 

2.2.4.2  Availability/Technical  Maturity 

2.2.5  Test  Cells 

2.2.5.1  Controlled  Variables 

2.2.5.2  Uncontrolled  Variables 

2.2.5.3  Sample  Sizes/Confidence  Levels/Technical  Significance 
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2.2.6  Test  Activities 

2.3  Test  Activity  1 

2.3.1  Relationship  of  Test  Activity  to  Program 

2.3. 1.1  Location 

2.3. 1.2  Instrumentation  Requirements 

2.3. 1.3  Resource  Requirements 

2.3. 1.4  Environmental  Considerations 

2.4  Test  Activity  n 

2.5  Schedule 

2.5.1  Test  Opportunities 

2.5.2  Program  Schedule 

3.0  Program  Analysis  Approach 

3 . 1  Overview  of  Planned  Analysis  Methods  and  Techniques 

3.1.1  Comparative  Analysis 

3.1.2  Statistical  Methods 

3.1.3  Other  Methods  and  Techniques 

3.2  Data  Analysis  Requirements 

3.2.1  Measures 

3.2.2  Variables 

3.2.3  Participants 

3.2.4  Test  Environments 

3.2.5  Systems  Operations  and  Interactions 

3 . 3  Analysis  of  Program  Issues 

3.3.1  Aggregation  of  Measures 

3.3.2  Resolution  of  Each  Program  Issue 

4.0  Program  Data  Management  Approach 

4 . 1  Data  Acquisition  Planning 

4.1.1  Test  Activity  Data  Requirements 

4.1.2  Criteria  for  Data  Collection  and  Analysis 

4.1.3  Instrumentation  Requirements 

4.1.4  Integrated  Data  Requirements  List  (IRDL) 

4.2  Data  Flow 

4.3  Data  base  Hardware  and  Software  (Connectivity  and  Acquisition) 

4.4  Data  Collection,  Handling,  and  Control 

4.4.1  On-site  Data  Management  Team 

4.4.2  Data  Transport  to  JTF  Library 

4.4.3  Data  Library 

4.4.4  Data  Entry/Storage 

4.5  Test  Activity  Reconstruction  and  Data  Validation 

4.6  Data  Reduction 

4.7  Analysis  Support 
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4.8  Analysis  Support  Products 

5 , 0  Program  Resources 

5 . 1  Resource  Management 

5.1.1  Planning 

5.1.2  Budgeting 

5.1.3  Funding 

5.1.4  Accountability 

5.2  Facilities  (Home  and  at  Test  Activity  Sites) 

5.3  Funding 

5.4  Personnel 

5 . 5  Equipment  and  Materials 

5 . 6  Service  Resources 

5.6.1  Army-OTP 

5.6.2  Air  Force-TRP 

5.6.3  Navy-Coordinate  with  POC 

5.6.4  Marine  Corps-Coordinate  with  POC 

6,0  JTF  Management 

6 . 1  Management  Overview 

6.2  JTF  Organization 

6.3  Staffing  and  Responsibilities 

6.3.1  Key  Staff 

6.3.2  Test  Activity  Teams 

6.4  Release  of  Information  (Data,  Test  Results,  and  Public  Relations) 

6.5  Milestone  Reviews 

6.6  Reports 

6.7  Training  Requirements 

6.8  Security 

6.9  Safety 

6.10  Environmental  Considerations 

7 . 0  Program  Documentation 

7.1  Plans 

7.2  Reports 

7.3  Program  Products 

7.3.1  Description  of  Product 

7.3.2  User  of  Products 

7.3.3  Disposition  of  Products 

7 . 4  Institutionalization  of  Products 


JTdiE  Handbook 
February  1, 1996 


L-3 


Annexes  (as  required  to  provide  rationale  used  in  the  selection  of  methodologies,  techniques,  test 
methods,  test  activities,  etc.). 

A.  Intelligence  and  Threat  Assessment 

B .  Operations  Security 

C.  Classified  Evaluation  Criteria 

D.  Detailed  Systems  Descriptions 

E.  Software  Evaluations 

F.  Human  Factors 

G .  Environmental  Considerations 

H .  Weather 

C .  EXAMPLE  OF  AN  OUTLINE  FOR  A  PROGRAM  LEVEL  DMAP 

I .  0  Introduction 

1.1  General 

1.2  Background 

1.3  Purpose 

1.4  Schedule 

2 . 0  Analysis  Approach 

2.1  Overview 

2.2  Analysis  Activities 

2.3  Test  Cell  Matrix 

2.4  Analysis  of  Issues 

2.4.1  Issue  1 
2.4.n  Issue  n 

2 . 5  Data  Requirements  Analysis 

2.5.1  Measures 

2.5.2  Test  Variables 

2.5.3  Participants 

2.5.4  Environments 

2.5.5  System  Operations  and  Interactions 

3 . 0  Data  Requirements 

3.1  General 

3 . 2  Integrated  Data  Requirements  List  (IDRL) 

4.0  Data  Sources  and  Instrumentation 

4.1  Test  Activity  1 

4.1.1  Data  Sources 

4.1.2  Instrumentation 

4.2  Test  Activity  n 

4.3  Unique  Instrumentation 
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5.0  Data  Collection,  Transportation,  and  Storage 

5 . 1  JTF  Data  Management  Functions 

5.1.1  Library 

5.1.2  Media  Identification  and  Control 

6.0  Test  Activities,  Data  Collection,  and  Management 

6.1  Media  Distribution,  Collection,  and  Control 

6.2  Security 

7 . 0  Data  Processing 

7.1  Test  Activities 

7.1.1  Quality  Assurance 

7.1.2  Quick  Look 

7.1.3  Reports 

7 . 2  JTF  Data  Management  Center  (Library) 

7.2.1  Data  Entry  and  Extraction 

7.2.2  Data  Reconstruction  and  Verification 

7.2.3  Data  Fusion 

7.2.4  Data  Control 

8.0  Data  Base  Structure 

8.1  Connectivity 

8.2  Hardware 

8.3  Software 

9.0  Analysis  Support  Products 

D .  EXAMPLE  OF  AN  OUTLINE  FOR  A  JT&E  FINAL  REPORT 

Executive  Summary 

ES.l  Introduction 

ES.2  Joint  Test  Approach 

ES.3  Major  Activities 

ES.4  Joint  Test  Schedule 

ES.5  Coordination  Chain  and  Responses 

ES.6  Conclusions  and  Recommendations 

ES.7  JTF  Products 

1.0  Purpose  and  Background 

1 . 1  Purpose  of  the  JT &E 

1 .2  Authorizing  Charter 

1.3  JT &E  Background 

1 . 4  Description  of  Concept  or  Systems  Tested 
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1 .5  JTF,  Test  Sites,  Locations,  and  Dates 

1 . 6  Classification  Statement 

2.0  JT&E  Concept  and  Description 

2. 1  Issues  and  Objectives 

2.2  Scope 

2.2.1  Missions 

2.2.2  Systems 

2.2.3  Assumptions  and  Limitations 

2.2.4  Participants 

2.2.5  Test  Focus 

2.3  Test  Methods 

2.4  Scenarios 

2.5  Decomposition  of  Issues 

2.5.1  MLMs 

2.5.2  MOEs 

2.5.3  MOPs 

2.6  Measures 

2.7  Data  Elements 

2.7.1  Baseline 

2.7.2  Types  of  Data 

2.7.3  Criteria  for  Data  Elements 

2.8  Test  Resources 

2.8.1  Exercise  Participation  and  Test  Activity  Sites 

2.8.2  Schedule 

2.8.3  Funding 

2.8.4  Proj  ect  Management 

2.8.5  Personnel  Support 

2.8.6  Equipment  and  Instrumentation 

3.0  Joint  Test  Results 

3 . 1  Summary  of  Joint  Test  Execution 

3.1.1  Data  Collection 

3.1.2  Analysis 

3.2  Issue  1 

3.2.1  Test  Activity  Method 

3.2.2  Results,  Findings,  and  Conclusions 

3.2.3  Recommendations 

3.3  Issue  n 

3.3.1  Test  Activity  Method 

3.3.2  Results,  Findings,  and  Conclusions 

3.3.3  Recommendations 
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4.0  Problems  and  Solutions 

5.0  Summary  of  JTF  Conclusions  and  Recommendations 

5 . 1  OSD,  Joint  Staff,  and  Service  Coordination/Concurrence 

5.2  Products 

5 . 3  Benefits  to  Participants 

5.4  Conclusions 

5 . 5  Recommendations 

Annexes  (as  required  to  substantiate  conclusions  and  recommendations) 

A.  Intelligence  and  Threat  Assessment 

B.  Operations  Security 

C.  Classified  Evaluation  Criteria 

D.  Detailed  Systems  Descriptions 

E.  Software  Evaluations 

F.  Human  Factors 

G .  Environmental  Considerations 

H.  Weather 

E.  EXAMPLE  OF  AN  OUTLINE  FOR  A  JTF  MANAGEMENT  REPORT 

Executive  Summary 
ES.l  Introduction 
ES.2  Joint  Test  Approach 
ES.3  Major  Activities 
ES.4  Lessons  Learned 

I. 0  Introduction 

1 . 1  Purpose  of  Report 

1.2  Program  Overview 

2.0  Assessment  of  Accomplishments 

2.1  Relative  to  First  Chartered  Mission 

2.2  Relative  to  Second  Chartered  Mission 

2.3  Relative  to  Third  Chartered  Mission 
2.n  Relative  to  n  Chartered  Mission 

3.0  Lessons  Learned 

3.1  Lessons  Learned  that  were  Documented  by  Previous  JTFs 

3.1.1  Lesson#! 

3.1.2  Lesson  #2 

3.1.3  Lesson  #3 
3.1. /I  Lesson  #n 
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3.2  Lessons  Learned  that  were  Unique  to  this  JTF 

3.2.1  Lesson  #1 

3.2.2  Lesson  #2 

3.2.3  Lesson  #3 
3.2.rt  Lesson  #n 

4.0  Conclusions  and  Recommendations 

4.1  Conclusions  (1-n) 

4.2  Recommendations  (1-n) 
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ANNEX  M 

TECHNICAL  ADVICE 


A.  INTRODUCTION 

The  JT&E  Program  receives  advice  from 
a  number  of  technical  organizations  that  are 
available  to  council  JT&E  Program  directors  and 
administrators,  Joint  Feasibility  Study  Directors, 
and  Joint  Test  Directors  on  technical,  test,  and 
doctrinal  issues  related  to  the  program  in  general 
and  specific  JT&Es  in  particular.  These  technical 
organizations  include  the  JT&E  Senior  Advisory 
Council  (SAC),  the  JT&E  Technical  Advisory 
Board  (TAB),  Technical  Advisory  Groups 
(TAG),  General  Officer  Steering  Committees 
(GOSC),  and  Functional  Expert  Panels.  The 
following  sections  contain  information  and 
recommendations  relative  to  when  an  advisory 
organization  could  be  beneficial,  who  is 
responsible  for  establishing/providing  advisory 
organizations,  and  how  these  advisory 
organizations  can  be  used  to  reduce  the  risk 
associated  with  JT&E  activities  and  events. 

B  .  JT&E  SENIOR  ADVISORY 

COUNCIL  (SAC) 

The  JT&E  SAC  is  a  senior-level  advisory 
group  that  reviews  candidates  for  entry  into  the 
JT&E  Program,  Joint  Feasibility  Study  and  Joint 
Test  and  Evaluation  progress  and  results,  and 
recommends  appropriate  actions  to  the 
D,T,SE&E.  The  Service  members  on  the  SAC 
also  commit  the  personnel  and  resources  required 
for  the  conduct  of  the  JT&E  activities.  The  SAC 
meets  in  June  of  each  year  or  at  the  call  of  the 
Chairperson.  The  following  organizations 
provide  personnel  who  are  permanent  members 
of  the  SAC.  A  list  of  current  SAC  members  is 


included  in  the  JT&E  Program  handout  entitled 
"Documents  Related  to  JT&E  Programs." 

The  JT&E  SAC  is  composed  of  the 
following  members: 

Co  chairperson:  D,T,SE&E  and  DOT&E 
OSD(PA&E) 

OSD(S&SS) 

Joint  Staff  (J-8) 

ODCSOPS-FDZ 

CNO(N-O91) 

OSD/C3I 

OSD/SOLIC 

USAFAL 

USMC  Systems 

Command-Chief  of  Staff 

BMDO,TMD 

Executive  Advisor:  Chairperson  of  JT&E 
Planning  Committee 
Technical  Advisor:  Chairperson  of  TAB 

(non-voting  member) 

The  responsibilities  of  the  JT&E  SAC  are 
as  follows: 

1.  Recommend  and  prioritize  candidates  for 

entry  into  the  JT&E  Program. 

2.  Review  JFS  results  in  terms  of: 

a.  Need  of  proposed  JT&Es. 

b.  Proposed  JT&E  concepts  and  Program 
Test  Designs. 

c.  Feasibility  of  proposed  JT&Es. 

d .  Impact  and  availability  of  resource 
requirements. 

e.  Availability  of  OSD  and  Service  funds  to 
support  the  proposed  JT&Es. 
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f.  Recommendations  for  lead  and 
participating  Services. 

g.  Anticipated  improvements  in  joint 
capabilities. 

h.  Opportunities  for  proposed  JT&Es  to  be 
incorporated  into  scheduled  joint 
activities. 

3.  Establish  schedules  for  JFSs  and  JT&Es. 

4.  Commit  Service  support  and  resources  to 
JFSs  and  JT&Es. 

5.  Review  and  recommend  JT&E  Program 
management  and  policy  changes. 

6.  Review  JT&E  final  reports. 

C .  JT&E  TECHNICAL  ADVISORY 
BOARD  (TAB) 

The  JT&E  TAB  is  an  organization  of 
senior  civilian  scientists  and  engineers  from  OSD 
and  the  Services  who  advise  the  SAC, 
nomination  sponsors,  FSDs,  and  JTDs  on  the 
technical  issues  related  to  the  JT&E  Program  in 
general  and  JFSs  and  JT&Es  in  particular.  The 
nomination  and  selection  of  TAB  members  is 
based  on  current  expertise  and  experience  relative 
to  the  planning,  conduct,  and  evaluation  of  large 
scale  tests,  exercises,  and  simulations.  The  TAB 
provides  advice  on  JT&E  nominations,  monitors 
JT&E  activities,  and  provides  technical  advice 
and  guidance  to  FSDs  and  JTDs  in  order  to 
minimize  the  risk  and  cost  associated  with 
JT&Es.  The  OSD  and  Services  nominate 
qualified  individuals  for  the  TAB.  The 

D, T,SE&E  reviews  the  nominations  for  JT&E 
TAB  membership  and  approves  those  considered 
to  be  most  qualified.  The  TAB.  meets  in  May  of 
each  year  or  as  directed  by  the  Chairperson.  A 
list  of  current  JT&E  TAB  members  is  included  in 
the  JT&E  Program  handout  entitled  "Documents 
Related  to  JT&E  Programs." 


The  JT&E  TAB  is  composed  of  the 

following  persons: 

Chairperson:  Rotated  periodically  among 
permanent  members 

Permanent  members:  One  member  from  each  of 
the  Services 

Non  permanent  members:  Civilian  scientists 
having  specific  technical  expertise 
related  to  a  particular  JT&E  and 
supported/participating  CINC 
representatives,  as  appropriate 

The  responsibilities  of  the  JT&E  TAB  are 

as  follows: 

1.  Review  and  determine  the  technical  adequacy 
of  APAs  and  PTPs 

2.  Review  and  determine  the  technical  adequacy 
of  individual  test  plans,  data  management  and 
evaluation  plans,  and  test  reports. 

3.  Provide  technical  recommendations  to  the 
JT&E  SAC  concerning  JT&E  Program 
candidates,  JFSs,  and  JT&Es. 

4.  Assist  in  defining  common  elements  among 
JT&Es,  maximizing  the  possibility  of 
resource  sharing,  minimizing  the  cost 
associated  with  JT&Es,  and  preserving 
significant  residual  assets  developed  in 
conjunction  with  JT&Es  for  future  use. 
These  residual  assets  are  primarily  scenarios, 
databases,  computer  simulations,  algorithms, 
test  facilities,  instrumentation,  and  threat 
simulators. 

5.  When  required,  recommend  personnel  to 
staff  TAGS  for  a  JFS  or  JT&E. 

6.  Provide  technical  advice  to  the  JT&E  SAC, 
JFSs,  and  JTDs. 

7.  Provide  recommendations  to  the  DDT&E 
relative  to  the  continuation  or  termination  of 
JFSs  and  JT&Es. 
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D.  TECHNICAL  ADVISORY 

GROUP  (TAG) 

A  TAG  is  a  technical  body  formed  to 
provide  direct  technical  support  to  a  JT&E.  The 
JTD,  with  the  coordination  and  approval  of 
DDT&E,  will  determine  if  and  when  a  TAG  is 
required.  TAG  members  may  be  nominated  by 
the  TAB,  suggested  by  the  Service  Operational 
Test  Agencies  (OTAs)  requested  by  the  JTD,  or 
suggested  from  other  sources.  The  JTD  will 
coordinated  directly  with  the  Service  OTAs  to 
obtain  the  services  of  TAG  members.  The 
composition  of  a  TAG  may  vary  as  the  technical 
nature  of  the  JT&E  changes.  The  TAG  will  meet 
at  the  call  of  the  JTD  and  is  not  a  permanent  full¬ 
time  part  of  the  JT&E  staff.  The  TAG  may 
provide  assistance  either  collectively  or  through 
the  actions  of  individual  members. 

A  TAG  is  composed  of  the  following 
members; 

If  established,  the  TAG  composition  will  be 
determined  by  the  JTD  and  will  normally  be 
made  up  of  individuals  from  the  Services 
(usually  senior  civilians)  who  have  technical 
proficiency  particularly  suited  to  the  JT&E . 

The  responsibilities  of  a  TAG  are  as 
follows: 

1 .  Review  and  provide  technical  comments  and 
recommendations  on  plans,  analysis  results, 
and  test  reports. 

2.  Assess  appropriateness  and  output  of 
simulations  suggested  or  developed  for  a 
proposed  JT&E  or  specific  test  events. 

3.  Identify  sources  of  technical  assistance  and 
provide  technical  liaison  with  these 
sources. 


4.  Provide  the  JT&E  TAB  with  technical  advice 
and  assistance  as  required. 

5.  Provide  the  JTD  with  technical  advice  and 
assistance  as  required. 

E.  GENERAL  OFFICER 

STEERING  COMMITTEE 
(GOSC) 

A  GOSC  should  be  established  to  review 
and  provide  advice  on  JT&E  issues  pertaining  to 
policy,  doctrine,  tactics,  and  resources.  A 
GOSC  is  a  group  of  general  officers  from  the 
Services  and  participating  theater  CINCs  who  are 
invited  by  the  JTD  to  advise  on  doctrine,  policy, 
or  tactics  issues.  The  intent  of  a  GOSC  is  to 
capture  and  integrate  Service  representation  at  a 
senior  officer  level.  The  JTD  will  determine  if 
and  when  a  GOSC  is  required.  GOSC  members 
may  be  nominated  by  the  TAB,  suggested  by  the 
Service  OTAs,  requested  by  the  JTD,  or 
suggested  from  other  sources.  The  JTD  will 
coordinate  directly  with  the  Service  OTAs  to 
obtain  the  services  of  GOSC  members.  The 
GOSC  will  meet  at  the  call  of  the  JTD  and  is  not 
a  permanent  full-time  part  of  the  JT&E  staff.  The 
GOSC  provides  guidance  either  collectively  or 
through  the  actions  of  individual  members  and 
should  be  included  in  the  review  cycle  of  JT&E 
produced  concepts,  products,  and  documents. 

A  GOSC  is  composed  of  the  following 
members: 

If  established,  the  GOSC  composition  will  be 
determined  by  the  JTD  and  will  normally  be 
made  up  of  flag  officers  (1-2  star  level)  from 
the  Services  involved  in  the  JT&E,  those 
providing  test  resources,  or  those  who  could 
be  affected  by  the  JT&E  results  and 
recommendations . 
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The  responsibilities  of  a  GOSC  are  as 
follows: 

1 .  Provide  the  JTD  with  a  forum  for  senior  level 
advice  and  guidance  with  respect  to  Service 
policy,  doctrine,  and  tactics. 

2.  Review  FTP,  test  results,  and  reports. 

3.  Provide  advice  relative  to  the  availability  or 
resources  and  Service  events  to  support 
planned  activities. 

4.  Provide  advice  relative  to  coordination 
requirements  and  the  identification  of 
signature  authorities. 

5.  Provide  assistance  in  arranging  meeting  and 
briefing  schedules. 

6.  Provide  assistance  in  obtaining  the  use  of 
Service  assets  and  resources  to  include 
information  or  access  to  classified  programs. 

F .  FUNCTIONAL  EXPERT 
PANELS 

If  the  JT&E  involves  issues,  techniques, 
or  equipment  that  are  technically  complex,  the 
JTD  may  want  to  consider  establishing 
Functional  Expert  Panels  to  obtain  expertise  or 
assistance  that  is  not  available  within  the  JTF. 
Functional  Expert  Panels  will  involve  the 
technical  communities  associated  with  the  JT&E 
activities,  and  this  involvement  will  be  helpful  in 
the  coordination  and  acceptance  of  JT&E  results. 
Functional  Expert  Panel  members  may  be 
suggested  by  the  TAB,  the  Service  OTAs,  other 
sources,  or  identified  by  the  JTD.  The  JTD  will 
coordinate  with  the  Service  OTAs  to  obtain  the 
services  of  Functional  Expert  Panel  members 
from  Service  resources.  The  JTD  will  coordinate 
with  the  OSD  JT&E  Coordinator  to  obtain  the 
services  of  non  DoD  Functional  Expert  Panel 
members. 

A  Functional  Expert  Panel  is  composed  of  the 
following  members: 


If  established,  the  Functional  Expert  Panel 
composition  will  be  determined  by  the  JTD 
and  will  be  made  up  of  members  who  are 
experts  in  the  required  specific  test  areas. 
Functional  Expert  Panel  members  can  be 
Service  members.  Government  Service 
employees,  contractors,  or  in  some  instances, 
civilians. 

The  responsibilities  of  a  Functional 
Expert  Panel  are  as  follows: 

1.  Provide  direct  technical  expertise  and 
assistance  to  the  JT&E. 

2.  Leverage  JTF  expertise  by  involving  the 
technical  communities. 

3 .  Assist  in  identifying  and  obtaining  additional 
expertise  when  required. 

4.  Expand  awareness  of  JT&E  efforts  to  the 
technical  communities. 

5.  Assist  in  coordination  and  acceptance  of 
JT&E  results  and  reports. 

6.  Assist  in  avoidance  of  the  "not  invented  here" 
syndrome. 

G .  JT&E  PLANNING  COMMITTEE 

(JT&E  PC) 

The  JT&E  PC  is  a  working  group  that 
assists  in  developing  and  evaluating  JT&E 
nominations  and  preparing  them  for  senior  level 
review.  The  JT&E  PC  Chairperson  is  the  OSD 
JT&E  Coordinator.  The  JT&E  PCs  permanent 
membership  is  drawn  from  OSD,  the  Joint  Staff, 
and  the  Services.  The  JT&E  PC  members 
interface  with  and  represent  their  respective 
Services  and  agencies.  The  Services  and 
agencies  may  send  additional  personnel  to  the 
JT&E  PC  meetings  to  provide  additional 
information  or  assistance  as  may  be  required. 
The  JT&E  PC  meets  at  the  call  of  the 
Chairperson  and  serves  as  the  action  staff  for  the 
JT&E  TAB  and  SAC. 
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The  JT&E  PC  is  composed  of  the 
following  members: 

Chairperson:  OSD  JT&E  Coordinator 
Permanent  members:  One  member  from  DOT&E 
One  member  from  the  Joint  Staff 
One  member  from  each  of  the 
Services 

One  member  from  OSD/P A&E 
Non  permanent  members:  One  representative 
from  each  agency  submitting  a 
nomination  for  consideration 


The  responsibilities  of  the  JT&E  PC  are 

as  follows: 

1.  Receive  and  prioritize  JT&E  Program 
nominations. 

2.  Review  JT&E  nominations  to: 

a.  Determine  jointness,  feasibility,  and 
validity  of  potential  results. 

b.  Identify  duplications  and  possible 
consolidations. 

c.  Determine  adequacy  of  data,  resource 
estimates,  and  test  objectives. 

3.  Recommend  nominations  by  priority  to  the 
TAB  for  consideration. 

4.  Functions  as  the  action  staff  for  the  JT&E 
TAB  and  SAC. 
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